ARI  Research  Note  2004-01 


U.  S.  ARMY  RESEARCH  INSTITUTE 
PROGRAM  IN  BASIC  RESEARCH 
2002  -  2003 


Basic  Research  Office 
Paul  A.  Gade,  Chief 


November  2003 


United  States  Army  Research  Institute 
for  the  Behavioral  and  Social  Sciences 


Approved  for  public  release;  distribution  is  unlimited. 

20031121  024 


U.S.  Army  Research  Institute 

for  the  Behavioral  and  Social  Sciences 

A  Directorate  of  the  U.S.  Army  Human  Resources  Command 


ZITA  M.  SIMUTIS 
Director 


NOTICES 

DISTRIBUTION:  This  Research  Note  has  been  cleared  for  release  to  the  Defense 
Technical  Information  Center  (DTIC)  to  comply  with  regulatory  requirements.  It  has 
been  given  no  primary  distribution  other  than  to  DTIC  and  will  be  available  only  through 
DTIC  or  the  National  Technical  Information  Service  (NTIS). 

FINAL  DISPOSITION:  This  Research  Note  may  be  destroyed  when  it  is  no  longer 
needed.  Please  do  not  return  it  to  the  U.S.  Army  Research  Institute  for  the  Behavioral 
and  Social  Sciences. 

NOTE:  The  views,  opinions,  and  findings  in  this  Research  Note  are  those  of  the 
author(s)  and  should  not  be  construed  as  an  official  Department  of  the  Army  position, 
policy,  or  decision  unless  so  designated  by  other  authorized  documents. 


REPORT  DOCUMENTATION  PAGE 

1.  REPORT  DATE  (dd-mm-yy) 
November  2003 

2.  REPORT  TYPE 

Final 

3.  DATES  COVERED  (from. . .  to) 

4.  TITLE  AND  SUBTITLE 

5a.  CONTRACT  OR  GRANT  NUMBER 

U.S.  Armv  Research  Institute  Proqram  in  Basic 

Research  2002  -  2003 

5b.  PROGRAM  ELEMENT  NUMBER 

61102 

6.  AUTHOR(S) 

5c.  PROJECT  NUMBER 

B74F 

Basic  Research  Office 

5d.  TASK  NUMBER 

5e.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

U.S.  Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences 

ATTN:  AHRC-ARI-BR 

5001  Eisenhower  Avenue 

Alexandria,  VA  22333-5600 

8.  PERFORMING  ORGANIZATION  REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

U.S.  Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences 

5001  Eisenhower  Avenue 

10.  MONITOR  ACRONYM 

ARI 

Alexandria,  VA  22333-5600 

1 1 .  MONITOR  REPORT  NUMBER 

Research  Note  2004-01 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  is  unlimited . 


13.  SUPPLEMENTARY  NOTES 


14.  ABSTRACT  (Maximum  200  words): 

This  document  contains  detailed  summaries  for  each  of  the  U.S.  Army  Research  Institute’s  basic  research 
contracts  for  the  fiscal  years  1998  -  2003.  These  summaries  are  grouped  according  to  three  Basic 
Research  Office  program  objectives;  Providing  fundamental  knowledge  to  improve  training  in  complex, 
digital  environments;  providing  fundamental  knowledge  to  improve  leader  assessment  and  accelerate 
leader  development;  and  providing  fundamental  knowledge  for  identifying  and  measuring  the  attributes 
and  skills  that  are  critical  to  soldier  recruiting,  performance,  and  retention  in  the  transforming  Army.  In 
addition  to  summarizing  what  was  done  or  is  being  done,  each  summary  also  describes  the  contributions 
of  that  research  effort  to  basic  behavioral  science  and  suggests  how  the  findings  might  benefit  the  Army 
and  other  military  services. 


SECURITY  CLASSIFICATION  OF 


16.  REPORT 
Unclassified 


17.  ABSTRACT 
Unclassified 


18.  THIS  PAGE 
Unclassified 


19.  LIMITATION  OF 
ABSTRACT 

Unlimited 


20.  NUMBER  21.  RESPONSIBLE  PERSON 

OF  PAGES  (Name  and  Telephone  Number) 

Paul  A.  Gade 
W  703/617-8866 


FOREWORD 


The  purpose  of  this  document  is  to  communicate  the  annual  progress  for  each  individual  research 
project  in  the  Basic  Research  Office  (BRO)  research  program  at  the  United  States  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI)).  The  summaries  contained 
herein  are  written  by  the  scientists  who  are  performing  the  work  to  provide  a  snapshot  of  their 
continuing  efforts.  In  addition,  BRO  conducts  a  more  detailed  in-progress  review  of  each  project 
each  year.  If  successful,  the  projects  within  BRO’s  basic  research  program  will  lay  the 
foundations  for  many  of  ARI’s  future  applied  behavioral  research  efforts. 
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THE  BASIC  RESEARCH  PROGRAM: 

Identifying  New  and  Promising  Technologies 


The  Basic  Research  Office  (BRO)  research  program  focuses  on  providing  the  the  basic  research 
underpinnings  for  the  personnel,  leader  development,  and  training  requirements  of  the  future. 
This  program  is  a  critical  link  between  the  world  of  behavioral  science  and  the  military 
community.  Searching  out  and  advancing  the  state-of-the-art  methods,  theories  and  findings  in 
behavioral  science,  encouraging  projects  most  likely  to  contribute  generalizable  scientific 
principles  and  new  knowledge,  and  supporting  those  efforts  that  have  potential  military 
relevance  and  likelihood  of  leading  to  applied  behavioral  technology  are  BRO’s  key  research 
goals. 

Universities  conduct  most  of  the  basic  research  program.  BRO  maintains  close  contact  with 
other  basic  behavioral  science  research  organizations  as  well  as  with  ARI’s  applied  researchers 
and  other  relevant  agencies  within  the  Army  and  other  military  services.  These  contacts  help  to 
define  issues  that  require  fundamental  research,  and  facilitate  the  transition  of  basic  research 
results  to  applied  programs  for  eventual  use  by  the  operational  Army. 

In  BRO’s  contract  programs,  a  Broad  Agency  Announcement  (BAA)  is  issued  each  year  to 
solicit  both  concept  papers  and  formal  proposals  relating  to  the  announced  program  research 
areas.  In  a  given  year,  the  BAA  highlights  the  research  objectives  of  special  interest,  and 
provides  an  open  call  for  proposals. 

On  the  following  pages,  the  reader  will  find  summaries  of  current,  on-going  BRO  contracts 
ranging  over  the  period  1998  -  2003.  There  are  three  current  BRO  program  objectives,  each  of 
which  will  be  discussed  in  detail  below.  They  are  as  follows: 

1 .  Provide  fundamental  knowledge  to  improve  training  in  complex,  digital  environments, 

2.  Provide  fundamental  knowledge  to  improve  leader  assessment  and  accelerate  leader 
development, 

3.  Provide  fundamental  knowledge  for  identifying  and  measuring  the  attributes  and  skills 
that  are  critical  to  soldier  recruiting,  performance,  and  retention  in  the  transforming 
Army. 

Training  for  Speed  in  Knowledge 

Basic  research  in  this  area  focuses  on  developing  concepts  and  methods  for  training  complex 
tasks  and  for  sustaining  complex  task  performance.  Assessing  the  cognitive  impact  of 
technology  requirements  arising  from  digital  systems,  semi-automated  systems,  and  robotic 
systems,  on  training  requirements  is  also  part  of  this  process.  One  of  the  efforts  in  this  area  seeks 
to  understand  the  impact  of  the  voluminous  multi-modal  data  on  performance  and  how 
individuals  and  teams  might  be  trained  to  integrate  and  use  such  rapidly  presented  data.  We  seek 
to  identify  unique  training  principles  and  methods  for  improving  interpersonal  skills  and  team 
adaptability  and  performance.  The  expected  outcome  is  applied  research  testing  the  principles 
and  methods  produced  in  this  work  package  in  Army  training  environments.  The  models  and 
theories  produced  should  be  useful  in  accounting  for  individual  differences  in  training  and 
facilitating  practical,  individualized,  adaptive  training  methods.  These  advances  will  translate 
into  improved  training  methods  in  a  wide  range  of  tasks. 


1 


Assessing  and  Improving  Leader  Skills 

Commensurate  with  the  requirements  for  rapidly  developing  adaptable,  flexible  leaders,  the  basic 
research  program  in  leader  development  is  directed  toward  providing  concepts  and  methods  for 
accelerating  leader  development  and  understanding  and  developing  leader  adaptability  and 
flexibility  in  a  manner  that  can  be  tested  in  the  applied  environment.  One  of  our  major  efforts  in 
this  area  uses  Sternberg’s  theory  of  practical  intelligence  to  develop  new  techniques  for 
acquiring  experience-based,  tacit  knowledge  as  a  means  for  rapidly  developing  effective  military 
leaders.  We  are  also  committed  to  discovering  and  testing  the  basic  cognitive  principles  that 
underlie  effective  leader-team  performance.  Understanding  the  dynamics  of  small  group 
leadership  in  face-to-face  and  distributed  team  environments  is  a  key  aspect  current  basic 
research  program  in  leadership.  For  example,  one  of  our  research  projects  seeks  to  develop 
leadership  techniques  that  foster  interaction,  communication,  and  trust  in  electronic 
environments.  Another  examines  the  development  of  team  leadership  behaviors  under  the  stress 
of  performing  in  a  metropolitan  hospital  shock  trauma  center  and  in  the  laboratory.  We  are 
completing  an  investigation  of  how  turnover  hinders  or  helps  team  performance.  Although 
turnover  can  be  harmful,  for  example,  when  a  team  loses  a  productive  member  and  must  expend 
time  and  energy  training  a  replacement;  turnover  can  also  be  helpful,  as  when  a  team  loses  an 
unproductive  member  and  gains  a  replacement  with  valuable  knowledge.  We  anticipate  that 
results  from  this  research  will  make  an  important  contribution  to  understanding  and  improving 
organizational  effectiveness. 

Pc  sonnel  Issues  for  the  New  Century 

Identifying  and  measuring  the  aptitudes  and  skills  that  are  unique  to  the  human  performance 
requirements  of  military  service  is  a  major  theme  of  this  basic  research  effort.  As  part  of  this 
process,  we  seek  to  devise  methods  that  assess  mental  flexibility  and  adaptability,  describe  how 
these  attributes  develop,  and  measure  their  contribution  to  performance  and  job  tenure.  For 
example,  we  have  begun  a  new  effort  to  assess  individual  mental  flexibility  since  mental 
flexibility  plays  a  critical  role  in  the  successful  application  of  the  mental  processes  needed  for 
adapting  to,  selecting,  and  shaping  the  environment  in  novel  situations.  Exploring  the 
sociological  and  psychological  factors  that  could  influence  recruitment,  retention,  and 
performance  is  part  of  this  work  package  as  well.  Applied  research  that  builds  the  understanding 
of  the  cognitive  processes  entailed  in  flexible  thinking  can  be  used  to  identify  those  who  are 
flexible,  adaptable  thinkers  and  to  develop  effective  instructional  programs  that  facilitate  the  use 
of  flexible  thinking  strategies  in  a  variety  of  contexts. 


This  document  provides  a  listing  and  brief  synopsis  of  ongoing  and  recently  completed  research 
efforts.  Project  listings  are  organized  into  the  three  aforementioned  research  objectives.  It  is 
important  to  note,  however,  that  this  program  is  but  one  of  many  programs  for  which  BRO  has 
responsibility.  Other  programs  in  BRO  include: 

•  Small  Business  Innovative  Research  (SBIR)  Program, 

•  Small  Business  Technology  Transfer  (STTR)  Program, 

•  International  Behavioral  Science  and  Technology  Watch, 

•  Graduate  student  apprenticeship  program  -  Consortium  Research  Fellows  Program  -  with  the 
Consortium  of  Metropolitan  Washington  Universities, 
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•  Outreach  efforts  to  Historically  Black  Colleges  and  Universities 
(HBCUs)  and  Minority  Institutions  (Mis),  and 

•  Research  support  in  behavioral  science  for  the  U.S.  Military  Academy. 

Additional  information  about  reports  from  these  research  efforts  is  available  upon  request. 
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BRO  RESEARCH  OBJECTIVE  #1: 

Provide  fundamental  knowledge  to  improve  training  in  complex, 
digital  environments. 


Research  under  this  objective  develops  concepts  and  methods  for 
training  complex  tasks  and  sustaining  complex  task  performance.  The 
focus  is  on  understanding  the  cognitive  impact  of  Future  Force 
technology  on  training  requirements,  the  impact  of  voluminous  multi¬ 
modal  data  on  performance,  and  developing  methods  for  improving 
the  interpersonal  and  cognitive  skills  that  affect  team  performance. 
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Training  For  Rapid  Interpretation  of  Voluminous  Multimodal  Data 

Contract#:  DAS  WO  1-02-K-0001  Contract  Dates:  5/1/02-9/30/04 

Institution:  Georgia  Institute  of  Technology  PI:  Dennis  J.  Folds 

Research  Summary 

The  overall  goal  of  our  research  is  to  increase  understanding  of  how  individuals  and 
small  teams  make  decisions  when  they  must  process  a  large  amount  of  information  very  rapidly. 
A  second  goal  is  to  determine  how  to  improve  this  ability  through  training.  We  are  studying  the 
effects  of  data  format  (photographs,  audio  files,  video  clips,  text  messages,  computer  graphs,  and 
map-based  icons)  on  rapid  decision  making.  We  are  also  studying  the  effects  of  overall 
information  volume  (i.e.,  the  total  number  of  information  items  that  a  person  must  process  before 
making  a  decision)  and  information  density  (i.e.,  the  number  of  information  items  relevant  to  the 
decision).  We  are  identifying  the  types  of  errors  that  people  commonly  commit  when  making 
decisions  in  this  manner,  and  are  studying  different  ways  to  improve  their  performance  through 
training. 

We  are  conducting  a  series  of  laboratory  experiments  with  individuals  and  small  teams.  The 
specific  objective  for  each  experiment  is  as  follows: 

•  Experiment  1  -  Assess  the  effects  of  data  format,  density,  and  overall  volume  on  decision 
making  performed  by  individuals. 

•  Experiment  2  -  Assess  effectiveness  of  anti-bias  training  for  individuals. 

•  Experiment  3  -  Assess  the  effects  of  data  format,  density,  and  overall  volume  on  decision 
making  performed  by  small  teams. 

•  Experiment  4  -  Assess  effectiveness  of  anti-bias  training  for  small  teams. 

The  participants  in  our  study  use  a  simulation  of  a  hand-held  computer  that  receives 
information  over  a  data  link.  The  information  is  in  the  form  of  a  video  clip,  a  photograph,  a  text 
message,  and  audio  file,  a  computer-generated  graph,  or  an  icon  on  a  map.  These  information 
items  arrive  in  rapid  succession  in  the  ‘inbox’  and  must  be  reviewed  by  the  participant.  They  use 
the  information  to  make  decisions  about  potential  incidents  (such  as  a  car  bomb)  that  they  are 
supposed  to  report.  The  information  they  receive  also  contains  a  lot  of  false  leads  that  should  be 
rejected.  We  developed  our  patterns  of  false  leads  so  they  would  tend  to  produce  the  types  of 
decision  errors  found  in  previous  research. 

Contributions  to  Basic  Research 

The  current  work  advances  basic  research  in  the  areas  of  naturalistic  decision  making, 
team  decision  making,  and  in  training  to  improve  decision  making.  Previous  research  has 
identified  characteristic  errors  (sometimes  called  biases)  that  humans  commit  when  making 
decisions  based  on  incomplete  or  conflicting  information.  Many  of  these  biases  seem  to  arise 
from  unconscious  strategies  that  reduce  the  effort  required  to  process  information,  thereby 
allowing  quick  decisions  to  be  made.  Our  research  is  examining  whether  these  same  types  (or 
perhaps  different  types)  of  errors  occur  when  an  individual  or  small  team  is  processing  large 
volumes  of  information  very  rapidly.  We  are  also  investigating  whether  the  same  cognitive 
processes  that  produce  the  biases  in  the  first  place  could  be  used  to  help  counter  the  bias,  through 
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training.  Specifically,  we  are  studying  whether  individuals  can  learn  to  recognize  the  patterns  of 
information  that  tend  to  produce  errors. 

Potential  Army/Military  Applications 

The  applications  of  this  research  for  the  Army  are  related  to  training  individuals  and 
small  groups  to  assess  information  received  in  the  field  and  to  make  rapid  decisions  based  on  that 
information.  While  in  the  field,  an  Army  Unit  of  Action  may  receive  information  over  a  data 
link  and  downloaded  to  a  hand-held  computer,  or  through  normal  voice  communications  over  the 
radio.  Specific  techniques  that  might  be  trained  include  spotting  the  ‘markers’  of  a  situation  that 
tends  to  produce  a  decision  error,  and  the  specific  countermeasures  to  take  once  the  markers  are 
spotted. 


Accomplishments 

We  have  completed  the  first  major  experiment  (Experiment  1 )  and  two  small-scale  pilot 
studies.  Our  first  small-scale  study  helped  us  refine  the  information  presentation  methods  and  to 
set  the  pace  of  information  presentation  so  that  subjects  would  have  just  enough  time  to  process 
all  the  information  and  to  make  rapid  decisions.  Our  second  small-scale  study  helped  us  develop 
the  division  of  responsibilities  for  a  three-person  team  and  structure  the  tasks  the  team  members 
will  perform.  Data  analysis  is  still  in  progress  for  Experiment  1 .  Preliminary  results  show  that 
our  participants  were  able  to  make  correct  incident  reports  for  about  50%  of  the  true  incidents, 
and  that  they  were  filing  reports  based  on  false  leads  for  about  25%  of  the  false  leads. 

We  have  also  completed  much  of  our  preparation  for  Experiment  2  and  are  conducting  a 
small-scale  pilot  study  that  will  help  us  refine  the  training  techniques  to  be  used  in  Experiment  2. 

Future  Plans 

During  the  next  year  we  plan  to  complete  Experiments  2  and  3  and  begin  work  on 
Experiment  4.  Experiment  2  will  examine  the  effectiveness  of  anti-bias  training  for  individuals. 
Individuals  will  be  trained  to  spot  the  markers  of  situations  that  tend  to  produce  errors,  and  to 
report  that  they  have  spotted  those  markers.  In  Experiment  3,  we  will  extend  our  research  to 
include  small  teams  (teams  of  three  individuals)  who  are  working  together  to  rapidly  process 
information  and  make  decisions.  In  Experiment  4  we  will  study  training  techniques  for  small 
teams. 
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Optimizing  The  Speed,  Durability,  And  Transferability  of  Training 


Contract#:  DASW01-99-K-0002  Contract  Dates:  8/1/99-7/31/03 

Institution:  University  of  Colorado  PI:  Alice  F.  Healy 

Co-PI:  Lyle  E.  Bourne,  Jr. 

Research  Summary 

The  U.  S.  Army  spends  much  time  and  many  resources  in  training  its  personnel.  Training 
is  essential  because  recruits  cannot  be  expected  to  come  equipped  with  the  military  knowledge 
and  skills  they  will  need  in  the  field.  But  training  is  costly,  so  it  is  important  to  insure  that  it  be 
accomplished  as  quickly  and  as  efficiently  as  possible.  Increasing  training  speed,  however, 
should  not  be  the  only,  or  even  the  most  important,  consideration.  If  soldiers  have  successfully 
learned  how  to  perform  a  task  during  training  but  then  forget  how  to  perform  it  at  the  time  that 
they  need  to  do  so,  the  training  has  clearly  been  inadequate.  Passing  a  test  at  the  end  of  training 
does  not  guarantee  later  success  in  the  field.  Training  needs  to  be  durable  as  well  as  efficient. 
But  even  durable  training  cannot  guarantee  that  learned  knowledge  and  skills  would  be  applied 
successfully  to  situations  different  from  those  encountered  during  training.  Training 
circumstances  can  rarely  capture  the  full  set  of  circumstances  under  which  tasks  are  encountered 
in  the  field.  It  is,  therefore,  essential  that  training  be  transferable  as  well  as  durable.  It  has  been 
the  aim  of  our  research  program  to  develop  principles  that  separately  optimize  the  three  major 
aspects  of  training:  (a)  speed  or  efficiency,  (b)  durability  or  long-term  retention,  and  (c) 
transferability  or  generalizability  to  new  situations. 

Contributions  to  Basic  Science 

The  many  research  directions  we  have  pursued  address  a  diverse  set  of  issues  but  have 
two  primary  goals.  First,  we  intensively  examined  extraneous  variables  (including  factual 
overload,  rapid  presentation  of  information,  stress,  frustration,  and  fatigue)  that  might  adversely 
affect  training  in  order  to  develop  procedures  to  counteract  their  deleterious  effects.  Second, 
building  on  previous  work,  which  revealed  that  long-lasting  knowledge  and  skills  are  highly 
specific  to  the  training  conditions,  we  conducted  a  series  of  studies  concerned  with  optimizing 
the  relationship  between  all  three  major  aspects  of  training  -  its  speed,  durability,  and 
transferability.  These  studies  share  a  common  analytical  and  experimental  methodological 
approach  and  the  common  theoretical  goal  of  understanding  the  psychological  principles 
underlying  the  acquisition,  retention,  and  generalization  of  knowledge  and  skills. 

Potential  Army/Military  Applications 

These  studies  also  share  the  common  applied  goal  of  improving  the  training  of  military 
personnel.  In  particular,  our  research  will  improve  training  techniques  to  enable  knowledge  and 
skills  to  be  acquired  quickly,  yet  be  still  accessible  across  long  delay  intervals  with  no  practice 
and  adaptable  to  new  situations  outside  the  training  environment.  The  balance  of  the  three 
aspects  of  training  (speed,  durability,  and  transferability)  is  not  necessarily  fixed  across  tasks  or 
even  within  a  given  task  but  rather  may  depend  on  a  variety  of  external  factors,  such  as  stress, 
frustration,  fatigue,  speed  of  information  presentation,  and  information  load,  that  can  change 
from  time  to  time.  Variations  in  any  one  of  these  factors  can  affect  the  interaction  of  these 
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aspects  of  training.  Hence,  our  studies  also  examined  the  three  aspects  of  training  under  various 
conditions  of  stress,  frustration,  fatigue,  speed  of  information  presentation,  and  information  load. 
This  examination  should  help  the  military  design  training  programs  that  will  allow  the  trainee  to 
contend  with  these  external  conditions  that  obtain  unpredictably  in  the  field  and  could  adversely 
influence  performance  if  not  properly  managed. 

Accomplishments 

Although  many  of  our  studies  have  overlapping  goals,  in  the  present  summary  we  focus 
on  a  subset  of  our  studies  and  discuss  two  separate  topics.  The  first  topic  involves  managing 
factual  overload,  rapidly  presented  information,  stress,  frustration,  and  fatigue,  with  an  emphasis 
on  tasks  involving  perceptual  and  motoric  processing.  The  experiments  reported  for  this  topic 
focus  on  the  specific  issue  of  initiating  and  executing  response  components  under  fatigue 
produced  by  prolonged  work.  The  second  topic  involves  optimizing  the  balance  of  the  three 
major  aspects  of  training.  The  study  reported  for  this  topic  focuses  on  ways  to  promote  transfer 
of  training. 

Managing  factual  overload,  rapidly  presented  information,  stress,  frustration,  and  fatigue. 
When  participants  work  continuously  over  time  on  a  task,  such  as  data  entry,  two  opposing 
processes  (facilitative  and  inhibitory)  might  affect  their  performance.  On  the  one  hand, 
performance  might  improve,  becoming  more  accurate,  faster,  or  both,  as  participants  master  the 
skills  required  of  data  entry.  On  the  other  hand,  either  or  both  aspects  of  performance  might 
deteriorate  as  participants  suffer  the  effects  of  fatigue  and  boredom  over  long  trial  periods.  We 
completed  two  experiments  addressing  fatigue  in  a  repetitive  data  entry  task.  Under  conditions 
promoting  fatigue,  participants  entered  four-digit  numbers  on  a  computer  terminal.  In 
Experiment  1,  accuracy  worsened  but  response  times  improved  both  across  and  within  session 
halves,  reflecting  an  increasing  speed-accuracy  tradeoff.  In  Experiment  2,  the  (largely  cognitive) 
time  to  enter  the  first  digit  of  each  number  improved  over  the  first  session  half  but  worsened  over 
the  second  half.  Accuracy  worsened  but  time  to  enter  the  remaining  digits  improved  across 
though  not  within  session  halves.  The  (purely  motoric)  time  to  press  the  enter  key  improved 
across  and  within  session  halves.  Thus,  through  a  combination  of  practice  and  fatigue, 
prolonged  work  affects  the  component  cognitive  and  motoric  processes  of  data  entry 
differentially  and  at  different  points  in  practice. 

An  additional  experiment  investigated  effects  of  articulatory  processing  on  number  data 
entry  (i.e.,  processes  involved  in  saying  or  subvocalizing  the  numbers).  Participants  entered  four¬ 
digit  numbers  presented  as  either  words  or  numerals  on  a  keyboard  either  under  articulatory 
suppression  or  in  silence.  The  articulatory  suppression  group  typed  initial  digits  faster  than  the 
silent  group,  but  for  subsequent  digits,  the  opposite  pattern  occurred  at  least  with  word  stimuli. 
Thus,  articulation  of  numbers,  which  promotes  entry  into  the  phonological  loop  of  working 
memory  (i.e.,  a  transient,  low-capacity  memory  store  for  previously  articulated  items),  retards 
processing  of  initial  digits  but  enhances  processing  of  subsequent  digits. 

Optimizing  the  balance  of  training  aspects.  Our  research  also  examined  training, 
retention,  and  transfer  of  a  duration  estimation  skill  using  an  arbitrary  unit  of  time  in  a 
prospective,  production  estimation  paradigm.  Specifically,  subjects  produced  time  intervals  on 
demand,  knowing  in  advance  that  they  would  be  required  to  do  so.  For  example,  they  were 
asked  to  indicate  by  clicking  a  response  key  when  32  time  units  had  elapsed  since  a  beep 
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sounded  (with  each  unit  equal  to  783  ms).  Participants  were  trained  with  feedback  and  then 
either  tested  immediately  for  transfer  without  feedback  or  retrained  with  feedback  1  week  later. 
There  were  three  training  and  retraining  conditions,  two  involving  secondary  tasks.  Retention  of 
the  duration  estimation  skill  was  perfect  across  the  1-week  delay  when  the  secondary  task 
condition  was  unchanged,  but  there  was  no  transfer  of  the  primary  duration  estimation  skill  when 
the  secondary  task  condition  was  changed.  These  findings  are  interpreted  within  the  procedural 
reinstatement  framework  (i.e.,  a  theoretical  framework  based  on  the  hypothesis  that  retention  and 
transfer  of  learned  knowledge  and  skills  depends  on  duplicating  the  learning  procedures  at  the 
time  of  test),  with  the  assumption  that  the  primary  duration  estimation  procedures  incorporate 
requirements  of  the  secondary  task. 


Final  Summary 

Our  research  program  aims  to  develop  principles  that  optimize  simultaneously  all  three 
characteristics  of  training  -  speed,  durability,  and  transferability  of  learned  knowledge  and  skills. 
Such  simultaneous  optimization  would  not  necessarily  optimize  any  one  characteristic  alone  but 
would  require  instead  a  balanced  consideration  of  all  three  characteristics.  The  balance  of  the 
three  aspects  of  training  is  not  fixed  across  tasks  or  even  within  a  given  task  but  rather  can 
depend  on  a  variety  of  external  factors,  such  as  stress,  frustration,  fatigue,  speed  of  presentation 
of  information,  and  information  load,  that  can  change  from  time  to  time.  The  studies 
summarized  illustrate  our  completed  work  on  two  topics.  The  experiments  reported  on  the  first 
topic  involve  a  data  entry  task.  They  focus  on  the  specific  issue  of  initiating  and  executing 
responses  under  fatigue  produced  by  prolonged  work.  These  experiments  demonstrate  that 
prolonged  work  affects  the  cognitive  and  motoric  processes  of  data  entry  differentially  and  at 
different  points  in  time.  The  experiments  reported  on  the  second  topic  involve  a  duration 
estimation  task,  which  is  in  some  cases  coupled  with  a  secondary  articulatory  suppression  task. 
They  focus  on  ways  to  promote  transfer  of  training.  These  experiments  demonstrate  that 
learning  how  to  estimate  durations  is  highly  specific  to  the  conditions  of  training  and  critically 
depends  on  whether  or  not  a  secondary  task  is  required.  These  studies  further  support  the 
theoretical  framework  for  understanding  training,  retention,  and  transfer  of  knowledge  and  skills 
that  we  have  called  “procedural  reinstatement.”  This  framework  not  only  accommodates  the 
results  of  our  extensive  series  of  laboratory  studies,  but  also  suggests  a  set  of  training  principles 
that  can  be  used  to  optimize  the  speed,  durability,  and  transferability  of  military  training. 
Specifically,  for  example,  the  following  pair  of  training  principles  are  suggested:  (a)  Reinstate 
the  conditions  of  study  as  closely  as  possible  when  taking  a  test  or  performing  in  the  field.  If  it  is 
possible  to  anticipate  test  or  field  conditions,  then  modify  training  conditions  to  match  them,  (b) 
To  be  effective,  training  must  incorporate  the  complete  set  of  field  task  requirements,  including 
all  secondary  tasks  likely  to  be  represented  in  the  field.  Instructors  and  course  planners 
sometimes  assume  that  teaching  a  primary  task  without  extraneous  requirements  will  benefit  the 
learning  process.  However,  if  such  secondary  task  requirements  exist  in  the  field,  then  use  of 
this  training  method  will  not  provide  optimal  transfer  to  field  performance.  This  effect  works 
both  ways.  That  is,  training  with  extraneous  secondary  task  requirements  will  not  be  optimal  if 
field  performance  does  not  include  those  requirements.  In  general,  learning  is  highly  specific  to 
the  conditions  of  training.  Thus,  for  optimal  performance,  the  entire  configuration  of  task 
requirements  during  training  needs  to  match  those  in  the  field. 
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Training  for  Efficient,  Durable,  and  Flexible 
Performance  in  The  Military 

Contract#:  DASW01-03-K-0002  Contract  Dates:  10/1/02-9/30/06 

Institution:  University  of  Colorado  PI:  Alice  F.  Healy 

Co-PI:  Lyle  E.  Bourne,  Jr. 

Research  Summary 

The  proposed  research  builds  on  our  earlier  studies  of  skill  training,  retention  of  trained 
skills  after  a  long  delay  interval,  and  transfer  of  trained  skills  to  novel  situations.  On  the  basis  of 
that  research  we  were  able  to  develop  a  set  of  training  principles  that  individually  optimize  the 
efficiency  and  durability  of  trained  performance.  But  we  have  also  discovered  that  conditions 
that  lead  to  durability  often,  and  perhaps  always,  lead  to  limited  flexibility  or  adaptivity.  In  fact, 
in  our  most  recent  research,  we  have  found  that  training  has  little  or  no  benefit  if  there  are 
discemable  differences  between  the  training  and  testing  situations  in  the  background  or  context, 
even  if  there  are  no  changes  made  in  the  primary  task  requirements.  The  focus  of  this  proposed 
project  is,  thus,  to  develop  training  procedures  for  teaching  knowledge  and  skills  that  will 
survive  primary  task  or  background  changes  and,  thereby,  produce  flexible,  as  well  as  efficient 
and  durable,  performance  in  military  tasks.  The  proposed  experiments  are  divided  into  three 
major  groups.  Experiments  in  the  first  group  are  designed  to  understand  how  individuals  can  be 
trained  to  contend  with  an  unpredictable  flow  of  information  often  large  in  quantity,  rapidly 
presented,  and  ambiguous.  Experiments  in  the  second  group  aim  to  identify  training  factors  that 
promote  adaptive  and  flexible  performance  in  the  field.  The  final  group  of  experiments 
examines  performance  in  dynamic  and  changing  task  environments.  We  conclude  with  an  effort 
to  create  a  complex  set  of  laboratory  tasks  similar  to  those  encountered  by  a  digitally  proficient 
pilot  operating  a  fully  computerized  cockpit,  and  not  unlike  those  of  the  digitally  proficient  “land 
warrior”  soldier  of  tomorrow.  The  major  aim  is  to  determine  the  extent  to  which  training 
principles,  first  established  in  a  simple  laboratory  task,  generalize  to  performance  under  these 
more  complicated  conditions. 


Contributions  to  Basic  Science 

We  have  proposed  the  following  theoretical  principles  for  optimizing  training:  principle 
of  contextual  interference,  training  difficulty  hypothesis,  depth  of  processing  principle, 
procedural  reinstatement  principle,  specificity  of  training  principle,  functional  task  principle,  and 
strategic-use-of-knowledge  principle.  The  primary  purpose  of  our  proposed  experiments  is  to 
collect  data  that  will  allow  us  to  accept  or  reject  each  of  these  training  principles  that  we 
formulated  on  the  basis  of  our  earlier  basic  laboratory  research  and  to  develop  and  test  new 
principles  that  are  supported  by  these  data. 

Potential  Army/Military  Applications 

Our  proposed  experiments  could  be  used  to  drive  applied  research.  To  illustrate  this 
potential  symbiosis  between  basic  and  applied  research,  we  give  two  brief  examples:  First,  our 
research  has  demonstrated  a  high  degree  of  specificity  from  training  to  subsequent  application. 
These  findings  have  crucial  implications  for  military  training  because  instructors  often  assume 
that  teaching  a  primary  task  without  extraneous  secondary  task  requirements  will  benefit  the 
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learning  process.  However,  our  findings  imply  that  to  be  effective,  training  must  incorporate  as 
many  as  possible  of  the  complete  set  of  field  task  requirements,  including  all  secondary  task 
requirements  that  might  be  imposed.  In  the  proposed  research,  we  will  test  the  boundary 
conditions  of  our  finding  using  a  variety  of  primary  and  secondary  tasks,  and  we  will  test 
hypotheses  concerning  ways  to  overcome  this  severe  specificity.  Although  the  tasks  we  are 
using  in  our  research  are  often  components  of  military  tasks  and  require  digital  proficiency,  they 
are  not  the  real  military  tasks  currently  being  trained  in  the  Army.  We  would  hope  that  applied 
research  units  would  be  interested  in  testing  whether  the  specificity  of  training  principle  we  have 
developed  would  apply  to  such  real  tasks  and  whether  the  methods  we  hope  to  develop  for 
overcoming  this  problem  could  be  adapted  to  improve  military  training.  To  that  end,  we  have 
had  discussions  with  Stephen  Goldberg  (Chief,  Simulator  Systems  Research  Unit)  about  future 
collaborative  research  on  the  applicability  of  our  training  principles  to  simulator  training. 
Second,  military  instructors  often  attempt  to  use  procedures  that  make  learning  easier  and  speed 
up  initial  skill  acquisition.  But  our  research  has  shown  that  introducing  sources  of  interference 
into  a  task  or  increasing  the  difficulty  of  the  task  slows  down  initial  skill  acquisition  but  leads  to 
ultimate  improvements  in  the  durability  and  flexibility  of  the  learned  skill.  In  our  proposed 
research,  we  will  test  this  principle  in  a  variety  of  task  domains,  and  we  will  test  an  extension  of 
this  hypothesis,  namely  whether  the  difficulty  of  a  secondary  task  has  a  similar  inhibiting  effect 
on  initial  learning  and  facilitating  effect  on  retention  and  transfer  as  does  the  difficulty  of  a 
primary  task. 


Accomplishments 

In  the  present  summary  we  focus  on  the  accomplishments  in  two  of  our  series  of 
experiments  that  have  helped  us  to  confirm  previously  proposed  training  principles  and  to 
identify  possible  new  principles. 

The  first  series  of  experiments  concerns  response  execution  in  a  motoric  coordination 
task  involving  various  forms  of  hand-eye  incompatibility  between  computer  mouse  and  monitor 
cursor.  We  found  that  practice  with  a  given  form  of  incompatibility  led  to  large  improvements 
during  training  and  high  durability  across  a  1-week  delay  but  minimal  transfer  from  one  type  of 
incompatibility  to  another.  Even  when  subjects  practiced  with  a  mouse  that  combined  up-down 
and  right-left  incompatibility,  there  was  no  transfer  to  performance  with  a  mouse  that  involved 
only  one  of  the  types  of  incompatibility.  Also,  training  with  the  more  difficult  right-left 
incompatibility  did  not  aid  in  performance  with  the  simpler  up-down  incompatibility.  The 
implications  for  military  training  is  that  when  eye-hand  coordination  is  involved  in  a  skill,  for 
practice  to  be  effective,  the  specific  forms  of  hand-eye  incompatibility  required  must  be  trained 
in  the  particular  way  that  will  be  encountered  subsequently  in  the  field. 

The  second  series  of  experiments  concerns  types  of  strategies  and  strategy  transitions 
during  training  of  a  classification  skill.  We  found  that  both  rule  difficulty  and  rule  salience 
determine  the  strategy  that  subjects  eventually  adopt  during  training.  In  the  present  case,  where 
the  same  simple  rule  was  used  in  all  tasks,  the  strategy  adopted  was  rule-based  or  instance-based, 
depending  on  the  salience  of  the  rule.  These  findings  imply  that  to  optimize  performance 
trainers  need  to  be  aware  of  both  the  salience  and  the  difficulty  of  the  operative  rule  for 
responding  in  any  given  task.  No  matter  how  simple  it  is,  when  a  rule  is  not  obvious,  learners 
may  adopt  a  strategy  in  which  they  memorize  specific  instances  rather  than  use  the  underlying 
rule. 
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Future  Plans 


Our  proposal  was  divided  into  three  substantive  sections,  with  each  section  further 
divided  into  three  subsections:  (a)  dealing  with  information  flow,  (b)  factors  promoting  adaptive 
and  flexible  performance,  and  (c)  coping  with  dynamic  environments  and  changing  task 
demands.  We  gave  a  detailed  description  of  at  least  one  major  experiment  in  each  of  these 
subsections.  During  the  first  year  of  the  project,  experiments  were  conducted  on  each  of  the 
three  substantive  sections;  three  major  experiments  were  undertaken  that  year,  one  in  each  of  the 
three  sections.  We  are  currently  analyzing  the  data  from  these  experiments,  and  the  data  analysis 
should  be  complete  by  the  end  of  the  first  full  year  of  the  contract  (i.e.,  September  30,  2003). 
The  specific  experiments  to  be  conducted  in  the  following  year  of  the  contract  will  depend  in 
part  on  the  outcome  of  these  data  analyses.  However,  we  plan  to  conduct  at  least  five  additional 
major  experiments  in  the  second  year  of  the  contract.  These  five  experiments,  like  the  three 
initial  ones,  will  represent  all  three  sections  of  the  proposal.  During  Year  2,  we  also  plan  to  write 
up  and  submit  for  publication  one  or  more  manuscripts  based  on  the  work  completed  in  the  first 
year  of  the  contract.  In  addition,  we  hope  to  extend  our  discussions  with  Dr.  Goldberg  about 
possible  collaborative  applications  of  our  research  to  military  simulator  training. 
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FOCUS:  A  Program  for  Training  Sensemaking  Skills 

Contract#:  1435-01 -01 -CT-31 161  Contract  Dates:  6/18/01-6/17/04 

Company:  Klein  Associates  Inc.  PI:  Gary  Klein 


Research  Summary 

The  main  objectives  of  this  research  effort  are  (a)  to  develop  a  model  that  describes  the 
cognitive  process  people  follow  in  order  make  sense  of  a  situation,  and  (b)  to  identify  training 
interventions  based  on  this  model  to  improve  individuals’  sensemaking  performance. 

Contributions  to  Basic  Science 

The  sensemaking  model  developed  in  this  effort  describes  a  higher-order  cognitive 
process  that  is  a  critical  component  of  decision-making  success  in  naturalistic  environments. 
Sensemaking  drives  the  ability  to  comprehend  the  meaning  and  relevance  of  large  amounts  of 
ambiguous  data.  This  ability  separates  the  individual  who  is  on  top  of  the  situation,  and  able 
to  form  expectancies  to  guide  smooth  performance,  from  the  individual  who  is  continually 
surprised  and  confused  and  acts  hesitantly  in  responding  to  the  crisis  of  the  moment.  This  effort 
contributes  to  basic  science  by  describing  this  critical  cognitive  process  and  exploring  ways  to 
improve  performance  of  this  process  through  training. 

Potential  Army/Military  Applications 

An  understanding  of  sensemaking  is  relevant  to  Army  efforts  to  design  various  types  of 
decision  support  systems.  By  describing  how  humans  actually  use  information  to  develop  and 
revise  interpretations  of  a  situation,  we  can  develop  command  and  control  and  other  technologies 
that  work  in  concert  with  the  human  and  support  his/her  natural  process  for  filtering  information 
and  building  and  maintaining  situation  awareness.  Our  findings-to-date  on  the  nature  of  the 
sensemaking  process  offer  initial  guidance  on  several  important  issues  in  the  field  of  decision 
support,  such  as  the  effect  of  information  rate  on  skilled  performance  and  possible  evaluation 
metrics  for  systems  intended  to  bolster  sensemaking  activities. 

Defining  the  process  of  sensemaking  also  appears  to  be  relevant  to  Army  training  needs. 
Our  findings  illustrate  areas  in  which  sensemaking  can  break  down  and  even  fail.  Training 
programs  can  be  structured  to  enable  practice  and  feedback  in  key  sensemaking  activities.  Our 
findings  also  highlight  areas  of  expert/novice  differences.  Training  scenarios  can  be  structured  to 
deliberately  build  sensemaking  expertise  in  non-experts  through  the  development  of  mental 
models,  fragmentary  mental  models,  and  anticipatory  thinking  skills. 

Accomplishments 

The  first  two  years  of  this  effort  have  produced  several  significant  findings  about  the 
nature  of  sensemaking.  First,  we  have  defined  sensemaking  as  the  process  of  fitting  data  into 
a  frame,  and  fitting  a  frame  around  the  data.  People  will  try  to  make  sense  of  data  inputs  they 
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receive  by  finding  or  constructing  a  story  to  account  for  the  data.  At  the  same  time,  their 
repertoire  of  stories  will  affect  which  data  elements  they  consider  and  how  they  will  interpret 
these  data.  Thus,  the  frame  and  the  data  work  in  concert  to  generate  an  explanation.  We  also 
suggest  that  data  elements  are  not  perfect  representations  of  the  world,  but  are  constructed.  They 
are  sampled  from  the  available  information  and  defined  in  terms  of  the  available  frames.  They 
are  abstractions  from  the  environment,  and  therefore,  distortions  of  reality.  We  assert  that  no 
more  than  three  or  four  of  the  key  data  elements  serve  as  anchors,  which  elicit  the  initial  frame 
used  to  sample  for  more  data  elements. 

In  our  Data/Frame  model  of  sensemaking  (Klein  et  al.,  2003),  we  have  differentiated  and 
described  six  activities,  or  building  blocks,  of  sensemaking:  elaborating  the  frame,  questioning 
the  frame,  preserving  the  frame,  comparing  frames,  seeking  a  frame,  and  re-framing.  Further,  we 
have  identified  two  sensemaking  cycles  that  encompass  these  activities:  the  assimilation  and 
accommodation  cycles,  corresponding  to  the  usage  suggested  by  Piaget  (1954).  During 
assimilation,  the  person  fits  data  and  information  into  the  frame  or  cognitive  map.  During 
accommodation,  the  person  alters  the  frame  to  fit  the  data. 

We  also  extended  our  discussion  of  quasi-logical  reasoning  (Klein,  Phillips,  Battaglia, 
Wiggins,  &  Ross,  2002)  into  the  notion  of  “possibility  theory,”  as  opposed  to  probability  theory 
and  deductive  inferences.  The  reasoning  we  observed  in  our  incident  data  rarely  employed 
deductive  logic.  Instead,  it  was  aimed  at  employing  inferences  in  order  to  identify  anchors  that 
could  be  used  in  sensemaking.  We  assert  that  quasi-logical  reasoning  and  possibility  theory  are 
strengths  in  the  process  of  sensemaking. 

Consistent  with  prior  research  (Barrows,  Feightner,  Neufeld,  &  Norman,  1978;  Chase 
&  Simon,  1973:  Elstein,  1989),  our  findings  also  suggest  that  expert/novice  differences  in 
sensemaking  performance  are  not  due  to  superior  reasoning  and  inferencing  strategies  on  the 
part  of  the  expert,  but  rather  to  the  quality  of  the  frame  that  is  brought  to  bear.  Experts  have  more 
factual  knowledge  about  their  domain,  they  have  built  up  more  experiences,  and  they  have  more 
knowledge  as  to  cause  and  effect  relationships.  They  are  more  likely  to  generate  an  accurate 
explanation  of  the  situation  than  novices  because  their  frame  enables  them  to  select  the  right  data 
from  the  environment,  interpret  it  more  accurately,  and  see  more  pertinent  patterns  and 
connections  in  the  data  stream. 

Our  research  also  suggests  that  developing  a  comprehensive  mental  model  for  a  complex, 
open  system  is  unrealistic.  Instead  most  people,  and  even  most  experts,  rely  on  fragments  of 
local  cause-effect  connections,  rules  of  thumb,  patterns  of  cues,  and  other  linkages  and 
relationships  between  cues  and  information  to  guide  the  sensemaking  process  (and  indeed  other 
high-level  cognitive  processes).  We  believe  that  a  set  of  fragmentary  mental  models  (FMMs) 
contribute  to  the  frame  that  is  constructed  by  the  sensemaker,  therefore  guiding  the  selection  and 
interpretation  of  data. 

We  have  elaborated  the  model  such  that  it  can  be  used  to  generate  testable  hypotheses — 
including  non-trivial  and  non-obvious  hypotheses — and  have  presented  several  hypotheses  as 
starting  points  for  model  testing. 
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Future  Plans 


In  Year  Three,  we  will  conduct  a  series  of  experiments,  including  a  training  approach  to 
boosting  sensemaking  skills  in  a  particular  domain.  Our  goal  will  be  to  test  some  of  the 
hypotheses  we  have  generated  based  upon  the  model. 

Specifically,  we  will  (1)  examine  the  effect  of  the  amount  of  information  on  sensemaking 
performance,  (2)  investigate  the  differences  between  Endsley’s  (1995)  model  of  situation 
awareness  as  a  state  of  knowledge,  and  the  Data/Frame  model  of  sensemaking  as  a  process,  and 
(3)  explore  a  training  approach  using  Cognitive  Task  Analysis  data  to  improve  sensemaking 
through  the  development  of  fragmentary  mental  models. 
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Understanding  Aspects  of  Individual  and  Collaborative  Skill  Acquisition  in 
Face-to-Face  and  Distance  Training  Situations 


Contract#:  DASW01-01-K0003  Contract  Date:  9/1/01-8/31/04 

Institution:  New  Mexico  State  University  PI:  Adrienne  Y.  Lee 

Co-PI:  Douglas  Gillan  & 

Nancy  Cooke 


Research  Summary 

The  purpose  of  this  research  is  to  examine:  a)  whether  teams  training  together  in  the  same 
location  (co-located)  will  train  more  quickly  and  efficiently  than  teams  who  are  not  in  the  same 
location  (distributed);  b)  whether  testing  in  the  same  situation/context  (co-located  to  co-located 
and  distributed  to  distributed)  is  better  than  training  in  different  situations/contexts  (co-located  to 
distributed  and  distributed  to  co-located);  c)  whether  we  can  develop  training  to  improve  the 
transfer  between  contexts;  and  d)  whether  the  training  will  improve  learning. 

Originally,  the  focus  was  on  Unmanned  Air  Vehicles  (UAV)  where  3  individuals  with 
different  jobs  all  must  fly  an  UAV  to  various  locations  and  take  pictures  of  specific  targets.  The 
new  focus  is  on  peacekeeping  missions  where  3  individuals  manage  different  resources  in  a 
Support  and  Stability  Operations  (SASO)  task.  Both  of  these  contexts  are  simulated  in  the 
laboratory  and  training/testing  is  performed  using  the  methodology  described  below. 

Contributions  to  Basic  Research 

In  recent  years,  the  advancement  of  military  technology  has  resulted  in  highly  complex 
skills  that  need  to  be  acquired  and  maintained  to  use  that  technology  (Barry  &  Runyan,  1 995). 
Consequently,  the  demand  for  ongoing  innovations  including  distance  training  has  increased. 
Distance  training  has  been  shown  to  be  extremely  cost  effective  and  has  produced  similar 
learning  outcomes.  Although  extensive  research  on  distance  learning  exists,  few  studies  have 
focused  on  group  (collaborative)  distance  training  delivered  by  intemet/web-based  technologies. 

A  parallel  development  has  occurred  in  the  area  of  team  training.  Even  though  group 
(collaborative)  learning  has  been  the  focus  of  many  studies  (Slavin,  1996)  and  progress  has  been 
made  on  knowledge  measurement  at  the  individual  level,  the  measurement  of  team  knowledge, 
and  team  cognition  in  general,  is  still  in  its  infancy  (Cooke,  Salas,  Cannon-Bowers  &  Stout, 
2000).  Thus,  research  on  how  to  improve  team  distance  learning  can  contribute  significantly  to 
the  field  in  several  areas  including  distance  education  and  across  context  team  transfer. 

Interest  in  distance  education  has  increased  due  to  a  concerted  effort  on  the  part  of  public 
educational  institutions  to  reach  new  populations  of  students  who  might  not  otherwise  be  able  to 
attend.  With  advancements  in  technology,  this  has  been  possible  (particularly  in  the  use  of  web- 
based  training).  On  the  other  hand,  few  controlled  studies  exist  to  determine  what  is  truly 
beneficial. 

Our  controlled  team  distance  training  research  will  also  contribute  to  the  transfer 
literature  generally.  Research  has  been  performed  on  individual  transfer  (Singley  &  Anderson). 
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Although  specifications  for  transfer  are  delineated,  research  on  context  effects  has  not  been  as 
conclusive  (see  McDaniel,  Anderson,  Einstein  &  O’Halloran,  1989  or  Wickens,  1987).  The 
current  research  focuses  on  describing  the  effects  of  a  particular  physical  context  change  on  team 
performance  (with  a  focus  on  cognitive  skills  but  also  some  measurement  of  social  skills). 

Potential  Army/Military  Applications 

The  applications  to  the  military  are  expected  to  be  in  training  principles  for  distance  and 
transfer  situations  (changing  from  one  situation  to  another).  Specifically,  these  studies  are 
designed  to  determine  what  conditions  promote  distance  training  and  transfer  and  what  training 
is  needed  to  improve  team  and  distance  transfer.  In  addition,  within  the  studies,  various 
assessment  measures  will  be  used  that  will  focus  on  individual  and  team  cognitive  and  social 
(social-cultural)  skill  development. 


Accomplishments 

In  the  Spring  and  Fall  2002,  the  first  study  on  co-located/distance  team  training  and 
transfer  was  performed  in  the  UAV  CERTT  lab.  Our  results  indicated  that  although  all  teams 
received  the  same  initial  declarative  knowledge  training,  distance  teams  outperformed  co-located 
teams  throughout  the  initial  hands-on  training  sessions  and  after  transfer.  The  distance  teams 
reached  a  plateau  but  this  was  probably  due  to  a  lack  of  feedback.  After  transfer,  all  teams 
experienced  a  decrement  in  performance  (similar  to  Singley  &  Anderson  ,  1989)  but  the  distance 
teams  (i.e.,  those  who  trained  at  a  distance  and  either  stayed  in  the  same  distance  condition  or 
transferred  to  a  co-located  condition)  showed  a  greater  improvement  for  the  two  training 
missions  after  transfer.  These  results  may  have  been  due  to  the  small  numbers  of  teams  or  the 
experimental  setup  in  the  UAV  lab  that  may  have  suppressed  co-located  performance.  We  are 
replicating  this  research  in  a  new  stability  and  support  operations  (SASO)  lab. 

Current  Research  and  Future  Plans 

For  Spring  and  Summer  2003,  the  focus  has  been  on  analyzing  the  data  collected  in  the 
UAV  CERTT  lab  and  determining  which  measures  are  most  useful  to  keep.  A  summary  of  the 
measures  used  and  inclusion  information  follows.  The  general  anxiety  measure  worked  well  but 
was  not  specific  enough  to  the  task.  The  trust  measure  also  worked  well  but  never  reached  high 
levels.  Because  these  results  indicate  that  developing  specific  team  trust  building  materials 
might  be  useful  for  the  second  study,  materials  used  in  pilot  testing  in  the  Spring  2002  are  being 
adapted  for  the  new  lab  studies.  The  first  experiment  included  many  knowledge  measures, 
including  one  that  was  similar  to  Pathfinder  measures.  These  measures  appeared  too  diffuse  and 
more  specific  measures  are  being  developed  for  the  second  study.  Many  other  team  measures 
were  used  in  the  first  study  based  upon  what  had  been  used  previously  in  the  CERTT  lab.  Of 
these  measures,  only  a  few  will  be  used  for  the  second  study.  No  measure  of  socio-cultural 
variables  was  used  in  the  first  study;  however,  for  the  second  study,  the  context  (SASO)  invites  a 
look  at  cross-cultural  beliefs  and  therefore  some  cross  cultural  measures  will  be  added.  In  the 
first  study,  the  bulk  of  the  questionnaires  came  at  the  end  of  the  experiment  (although  the 
knowledge  assessments  were  distributed).  In  the  second  experiment,  these  questionnaires  will  be 
distributed  throughout  the  experiment. 
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The  main  focus  in  the  late  spring  and  early  summer  has  been  on  the  building  of  the  lab 
and  testing  of  software.  The  current  full  implementation  of  the  software  is  well  understood  and 
we  a  requesting  changes  from  Aptima  in  order  to  run  the  second  study.  In  addition,  tutoring 
systems  are  being  developed  to  provide  initial  information  and  team  training  before  the  SASO 
missions  are  run  (using  the  software).  The  lab  should  be  ready  for  the  experiment  by  the  end  of 
August. 
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Exploring  the  Interaction  of  Implicit  and  Explicit  Processes  to 
Facilitate  Individual  Skill  Learning 

Contract  #:  DAS  WO  1 -00-K-00 1 2  Contract  dates:  9/00-8/04 

Institution:  University  of  Missouri-Columbia  PI:  Ron  Sun 

Co-PI:  Robert  Mathews 


Research  Summary 

Any  high  level  of  skill  depends  on  both  conceptual  and  subconceptual  (experiential) 
knowledge.  However,  experts  are  often  only  aware  of  their  explicit  conceptual  knowledge. 
Experientially  acquired  subconceptual  knowledge  is  more  akin  to  pattern  recognition.  For 
example,  when  you  recognize  a  friend’s  face,  you  instantly  know  who  the  person  is,  but  you  may 
not  be  aware  of  what  cues  or  features  are  being  used  to  recognize  him/her.  This  lack  of 
awareness  of  essential  experiential  knowledge  creates  serious  challenges  for  training  and 
learning  in  the  military  as  well  as  civilian  context.  The  purposes  of  this  research  program  are  (1) 
to  experimentally  investigate  the  interaction  between  knowledge  gained  from  hands-on 
experiential  (implicit)  learning  and  explicitly  formulated  conceptual  knowledge  in  the 
development  of  skills  and  expertise,  (2)  to  clarify  how  conscious  reflection  and  resulting 
conceptual  knowledge  can  be  employed  to  enhance,  rather  than  interfere  with,  experiential 
learning,  (3)  to  discover  the  optimal  mix  of  experiential  and  conceptual  training  over  a  range  of 
skill  acquisition  from  early  novice  to  a  high  level  of  expertise,  and  (4)  to  develop  computational 
models  and  theories  of  human  skill  learning  that  emphasize  the  interaction  of  these  two  types  of 
learning,  which  may  be  of  use  in  structuring  and  optimizing  training,  on  the  basis  of  our 
experimental  and  simulation  results. 

Contributions  to  Basic  Research 

The  current  work  advances  basic  research  in  the  areas  of  learning  and  cognition.  One 
product  of  this  effort  is  a  comprehensive  conceptual  framework  describing  the  ways  these  two 
types  of  knowledge  interact  to  produce  expertise  in  tasks  that  require  both  speed  and  accuracy. 
This  framework  suggests  that  performance  can  be  controlled  by  either  a  connectionist  knowledge 
base  (implicit  mode)  or  application  of  a  symbolic  conceptual  mental  model  (explicit  mode). 
Implicit  control  is  fast  but  prone  to  error,  particularly  in  early  levels  of  skill  acquisition.  Explicit 
control  is  more  accurate  but  slow  to  apply,  and  prone  to  loss  by  forgetting  over  a  retention 
interval.  We  have  found  that  reflection  about  how  one  is  performing  the  task  can  be  very 
beneficial  following  short  periods  of  practice.  However,  it  is  often  effective  when  learners  are 
provided  hints  that  direct  their  reflection  in  productive  directions. 

A  computational  cognitive  architecture,  CLARION,  markedly  different  from  other 
existing  cognitive  architectures,  is  developed  in  this  work  to  simulate  and  capture  a  wide  range 
of  quantitative  data  that  are  related  to  the  interaction,  based  on  the  above  ideas.  This  will  help  us 
to  predict  (and  ultimately  optimize)  training  and  learning  processes.  We  carry  out  simulation 
experiments  in  the  domain  of  process  control  tasks,  as  well  as  the  domains  of  some  other  tasks, 
and  generate  new  insight  and  interpretations  that  can  further  explicate  the  interaction  between 
implicit  and  explicit  processes.  These  anticipated  outcomes  (data,  models,  and  theories)  would 
provide  a  clearer  and  more  comprehensive  perspective  on  skill  learning.  Our  models  and 
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theories  will  be  useful  in  better  understanding  skill  learning  (as  well  as  in  helping  to  improve 
learning  processes).  Our  models  and  theories  may  also  be  useful  in  understanding  individual 
differences  in  skill  learning. 


Potential  Army/Military  Applications 

It  is  expected  that  applications  of  this  research  for  the  Army  will  be  in  the  area  of  training 
and  human  performance.  A  basic  problem  for  the  Army  is  how  to  ensure  that  novices  in  a 
Military  Occupational  Specialty  (MOS)  move  quickly  to  achieve  more  advanced  performance 
(and  perhaps  to  expertise)  as  a  result  of  their  training.  In  addition,  most  training  focuses  on 
teaching  conceptual  knowledge  rather  than  setting  up  the  opportunity  for  substantial  experiential 
knowledge.  While  this  may  be  appropriate  for  some  specialties,  many  specialties  involve 
working  with  complex  systems  that  are  better  learned  initially  through  extensive  experience  than 
with  lectures  or  textbook  lessons.  As  service  personnel  gain  in  expertise,  it  may  often  become 
appropriate  to  focus  more  on  conceptual  knowledge.  Our  research  will  lead  to  some  initial 
guidelines  about  1)  what  types  of  training  are  most  appropriate  for  learning  complex  systems,  2) 
what  types  of  training  are  most  appropriate  for  a  given  level  of  experience,  3)  how  MOS  experts 
might  best  communicate  their  knowledge  to  MOS  novices,  and  4)  individual  differences  in 
training  performance  and  how  these  can  be  explored  to  enhance  learning  (or  in  other  words, 
developing  individualized,  adaptive  training).  We  would  be  interested  in  talking  with  our  liaison 
officers  about  implementing  and  transitioning  these  ideas  in  the  future. 

Accomplishments 

We  completed  a  large  series  of  experiments  in  two  different  task  domains:  artificial 
grammar  learning  and  process  control.  Both  tasks  involve  learning  a  system  that  operates 
according  to  complex,  difficult  to  learn  rules.  In  the  artificial  grammar  experiments,  participants 
learn  to  spot  poison  can  labels  on  a  computer  simulation  of  a  starship  that  has  been  invaded  by 
enemy  agents.  Identification  of  poison  food  labels  requires  learning  to  identify  sequences  of 
letters  generated  by  a  finite  state  grammar.  Training  for  this  task  might  include  memorizing  a 
diagram  of  the  grammar  (explicit  training),  memorizing  cases  (implicit  training),  or  an  integrated 
training  that  involves  tracing  cases  through  the  grammar  diagram.  Principal  findings  include 
slow,  but  accurate  responding  when  explicitly  trained  and  fast,  but  less  accurate  responding 
when  implicitly  trained.  Integrating  or  mixing  types  of  training  generally  produced  performance 
more  accurate  than  implicit  training  and  faster  than  strict  explicit  training.  Simulations  using 
CLARION  have  confirmed  these  findings.  One  interesting  finding  was  that  when  exposed  to 
both  types  of  training,  participants  showed  a  tendency  to  prefer  using  the  implicit  mode.  In  the 
case  of  strict  explicit  followed  by  implicit  training  (a  pattern  that  is  common  in  many  training 
situations,  e.g.,  explicit  schooling  followed  by  field  training)  we  noticed  a  loss  of  accuracy  as 
learners  shifted  toward  the  implicit  mode.  We  think  training  can  be  accelerated  and  this  post¬ 
training  drop  eliminated  by  using  the  explicit  conceptual  knowledge  to  prime  rather  than 
compete  with  implicit  learning.  This  notion  is  being  tested  in  new  experiments. 

The  process  control  task  involves  learning  to  control  the  temperature  of  a  simulated 
nuclear  reactor  by  controlling  the  number  of  fuel  pellets.  This  task  is  known  to  be  difficult  to 
leam  and  difficult  to  explain  how  one  accomplishes  the  task  when  it  is  learned.  The  results  show 
that  detailed  reflection  during  task  performance  can  interfere  with  learning.  However,  reflection 
following  short  periods  of  practice  can  be  beneficial,  especially  when  hints  are  provided  about 
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how  to  reflect.  Unlike  the  artificial  grammar  task,  there  are  large  individual  differences  in  the 
ability  to  learn  this  task.  Our  participants  were  asked  to  formulate  explicit  rules  or  policies  for 
performing  the  task  as  they  were  learning.  On-going  analyses  of  these  policies  are  expected  to 
reveal  why  many  participants  can  never  leam  to  control  the  reactor.  Simulations  using 
CLARION  are  also  producing  interesting  hypotheses.  Potentially,  we  can  facilitate  learning  in 
these  “failing”  participants  by  instructing  them  on  what  to  focus  on  when  they  reflect. 

In  the  process,  the  cognitive  architecture,  CLARION,  is  being  refined.  It  is  shaping  up  to 
be  a  rather  comprehensive  framework  for  modeling  and  understanding  skill  learning  in  a  variety 
of  circumstances.  Simulations  using  CLARION  have  provided  interesting  interpretations  in  a 
number  of  task  domains  and  more  simulations  are  under  way. 

Future  Plans 

Given  the  limited  amount  of  funding  left  for  our  project,  we  will  be  focusing  mainly  on 
analyzing  gathered  data  and  finishing  off  the  CLARION  simulation  software  development.  We 
will  also  be  working  on  a  few  major  articles  summarizing  and  analyzing  our  results.  We  also 
have  plans  for  some  additional  interesting  human  experiments  and  further  cognitive  architecture 
development.  These  latter  studies  will  need  to  be  carried  out  with  additional  funding  in  the 
future. 
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BRO  RESEARCH  OBJECTIVE  #2: 


Provide  fundamental  knowledge  to  improve  leader  assessment  and 
accelerate  leader  development. 


Research  that  falls  under  this  heading  provides  concepts  and  methods  for 
accelerating  leader  development,  understanding  and  developing  leader  adaptability 
and  flexibility,  and  discovering  and  testing  the  basic  cognitive  principles  that 
underlie  effective  leader-team  performance.  Understanding  the  dynamics  of  small 
group  leadership  in  face-to-face  and  distributed  team  environments  is  critical  to 
this  research  objective  as  well. 
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Interactivity,  Communication,  and  Trust: 
Implications  for  Leadership  and  Training  in  the  Electronic  Age 

Contract  #:  DASW01-00-K-0015  Contract  Dates:  9/30/00  -  10/1/03 

Institution:  University  of  Arizona  PI:  Judee  K.  Burgoon 

Co  Pi’s:  Suzanne  Weisband  & 
Joseph  Bonito 


Research  Summary 

Successful  leadership  and  team  performance  are  built  on  a  foundation  of  trust  and 
effective  communication  between  and  among  leaders  and  team  members.  New  communication 
technologies  such  as  email,  text  chat,  cell  phones,  videophones,  and  teleconferencing  are  now 
pervasive  in  today’s  military  as  leaders  and  their  teams  now  often  work  at  a  distance  from  one 
another.  The  1 3  laboratory  and  field  experiments  in  our  research  program  to  date  seek  to  answer 
the  question  of  how  these  new  technologies  affect  a  leader’s  ability  to  foster  high  trust,  morale, 
and  performance  within  his/her  team.  The  guiding  explanatory  principle  under  test  is  the 
principle  of  interactivity:  whether  messages  sent  and  received  are  coherently  and  tightly  linked, 
create  coordinated  communication,  and  are  marked  by  involvement,  mutuality  (sense  of 
connection,  receptivity,  common  ground,  mutual  understanding),  and  individuation  (clear  and 
detailed  knowledge  of  sender  and  receiver  identities).  Technologies  that  enable  proximal,  real¬ 
time,  and  multi-sensory  message  exchange  usually  promote  interactivity.  The  current  round  of 
investigations  continues  to  address  what  forms  of  electronic  communication  help  or  hinder 
interactivity.  It  also  considers  whether  task  load — the  degree  to  which  a  task  is  cognitively 
and/or  physically  effortful  and  demanding — alters  interactivity  and  trust.  Results  speak  to  best 
communication  practices  that  will  help  leaders  maximize  trust  when  trust  is  needed,  dampen 
interactivity  and  trust  when  skepticism  is  needed,  and  prevent  unintended  negative  consequences 
of  using  electronic  media.  We  offer  guidelines  for  leaders  in  selecting  and  managing 
communication  processes  and  principles  for  training  soldiers  in  their  use. 

Contributions  to  Basic  Research 

This  program  of  research  contributes  to  knowledge  and  theorizing  in  the  domains  of 
leadership,  group  performance,  trust,  and  human  communication.  It  has  direct  implications  for 
current  organizational  theories  that  address  distributed  "virtual"  processes  for  accomplishing 
work  as  well.  As  work  becomes  more  global  and  distributed,  the  whole  nature  of  organizing  will 
inevitably  be  modified.  Such  changes  need  to  be  guided  by  systematic  empirical  investigations 
rather  than  by  anecdote  and  personal  experience,  which  in  many  quarters  form  the  primary 
source  of  “data.”  The  theoretical  principal  framing  our  series  of  controlled  experiments  offers 
one  coherent  account  for  how  new  technologies  utilized  for  distance  communication  impact 
communication  patterns,  trust,  and  leadership.  The  principle  of  interactivity  proposes  that  the 
greater  the  degree  of  interactivity,  the  greater  the  trust.  Where  high  trust  is  warranted,  more 
interactivity  should  result  in  better  performance.  However,  where  skepticism  and  suspicion  are 
warranted,  such  as  when  the  task  involves  recognizing  and  deciphering  invalid  information  or 
uncovering  ulterior  motives,  greater  interactivity  may  foster  too  much  trust  and  result  in  poor 
performance.  Degree  of  interactivity  is  therefore  a  key  factor  in  predicting  team  trust  and 
performance. 
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Our  research  program  offers  the  richest  explication  in  the  scholarly  literature  of  what 
constitutes  interactivity  and  its  possible  effects.  Original  contributions  include  the  focus  on 
message  exchange  itself,  and  introduction  of  the  multidimensional  qualities  of  involvement, 
mutuality,  and  individuation  as  primary  markers  of  interactivity.  Our  assessment  of  related 
communication  qualities  also  offers  new  ways  to  gauge  whether  a  team  will  develop  and 
maintain  the  desired  level  of  trust  and  performance.  The  use  of  both  laboratory  and  field 
experiments  and  the  use  of  a  variety  of  tasks  and  forms  of  task  load  also  increase  our  ability  to 
generalize  this  research  to  other  contexts  and  to  discover  those  conditions  under  which  the 
general  principle  may  not  hold.  A  final  contribution  is  the  development  of  a  new  web-based 
task,  StrikeCom,  which  can  be  used  for  testing  various  forms  of  task  load,  group  composition, 
and  deceit,  among  other  research  objectives.  StrikeCom  has  the  virtues  of  engaging  participant 
interest,  similating  the  kinds  of  tasks  soldiers  might  confront  and  having  the  flexibility  to  modify 
such  features  as  different  map  overlays,  availability  of  a  common  situational  awareness  space, 
grid  size,  number  of  players,  number  of  information  assets,  and  reliability  of  the  information  that 
is  returned. 


Potential  Applications 

Research  under  the  preceding  contract  had  established  that  trust  and  credibility  are 
strongly  influenced  by  the  degree  of  interactivity  during  communication  and  that  the 
communication  process  is  itself  affected  by  features  of  the  technology  in  use.  The  implication  is 
that  leaders  may  select  technologies  that  foster  the  desired  level  of  interactivity  and/or  arrange 
conditions  to  ameliorate  any  drawbacks  of  technologies  they  are  required  to  use.  The  research 
also  provided  evidence  that  frequent  and  explicit  awareness  of  communication,  especially  during 
the  early  stages,  improves  team  performance.  The  implication  is  that  when  leaders  model  and 
promote  the  exchange  of  task  information  and  awareness  of  others,  team  members  may  be  more 
motivated  to  work  hard  on  behalf  of  the  team.  This  further  confirms  the  value  of  interactivity  in 
achieving  desired  outcomes,  in  that  awareness  of  the  communication  process  and  one  another  are 
direct  evidence  of  the  involvement,  mutuality,  and  individuation  that  marks  interactivity. 

Army  communication  training  for  soldiers  and  leaders  must  include  preparation  for 
communicating  in  mediated  settings.  Leaders  can  be  trained  to  analyze  what  degree  of 
interactivity  is  needed  for  a  given  task  or  type  of  communication  and  to  select  the  appropriate 
communication  medium  for  the  type  of  communication  taking  place.  They  can  also  be  trained  to 
be  observant  about  qualities  of  the  communication  that  is  taking  place  between  themselves  and 
their  team  members  and  among  team  members.  Subordinates  can  likewise  gain  heightened 
awareness  of  how  mediated  communication  affects  team  trust  and  performance.  If  morale,  trust, 
or  performance  appears  to  be  waning,  they  can  be  trained  in  techniques  to  increase  interactivity. 

Protocols  can  also  be  developed  for  ensuring  frequent  and  predictable  interaction. 
Software  tools  can  be  adopted  to  aid  this  process.  Protocols  can  also  be  developed  that  specify 
the  “best  practices”  in  terms  of  matching  communication  and  information  tools  to  desired 
objectives  and  predicted  levels  of  task  load.  Such  protocols  can  identify  potential  pitfalls  to 
avoid  or  compensatory  efforts  that  can  be  used  so  that  negative  consequences  such  as 
noncompliance,  distrust  and  miscommunication  do  not  occur  under  conditions  of  high  time 
pressure,  cognitive  effort,  inalertness,  or  fatigue.  For  example,  periodic  FtF  communication  may 
be  recommended,  or  a  shift  from  text  to  voice  communication  may  be  advised.  Other  potential 
applications  are  the  development  of  software  that  will  create  a  more  visible  form  of  social 
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awareness  for  team  members  and  leaders  who  cannot  communicate  face-to-face  so  that 
collaborative  work  does  not  suffer. 


Accomplishments 

The  current  investigations  were  undertaken  to  replicate  and  extend  previous  findings  by 
using  a  variety  of  tasks  conducted  under  several  forms  of  task  load.  In  the  first  experiment,  team 
members  completed  business  case  studies  under  low,  moderate,  or  high  time  pressure  while 
communicating  face-to-face  (FtF)  or  via  audio  or  video  computer-mediated  communication 
(CMC).  The  quality  of  members’  communication  and  its  interactivity  was  measured  on  13 
different  indices,  such  as  being  involved  and  receptive  to  each  other’s  ideas,  mutual 
understanding  and  “connection,”  and  having  a  pleasant,  coordinated  and  smooth-running 
discussion.  Outcomes  included  decision-making  effectiveness  and  members’  ratings  of  one 
another  on  trust.  Audio  communication  emerged  as  superior  for  fostering  interactivity  and  other 
favorable  communication  qualities  (e.g.,  rich  yet  efficient  information  exchange,  task  focus,  and 
positivity).  Time  pressure  also  made  a  difference:  the  higher  the  pressure,  the  less  rich  and 
analytical  the  communication,  but  the  greater  the  coordination.  Moderate  time  pressure  was  best 
for  keeping  tension  and  pleasantness  at  an  optimal  level.  However,  time  pressure  degraded 
decision-making  effectiveness,  especially  in  the  audio  condition,  suggesting  that  with  increased 
time  pressure  comes  the  need  for  more  visual  cues  to  coordinate  task  performance  and  gain 
feedback.  The  various  modalities  were  equally  effective  in  establishing  and  maintaining  trust. 
These  results  imply  that  leaders  would  gain  benefits  from  using  cell  phone  and  teleconferencing 
forms  of  communication  as  long  as  task  load  is  modest,  but  should  expect  process  and 
performance  losses  if  time  pressure  increased. 

The  second  experiment  entailed  a  “get-acquainted”  task  during  which  half  of  the  teams 
had  a  member  who  gave  deceptive  answers.  This  was  considered  a  form  of  task  load  in  that 
naive  partners  needed  to  be  alert  to  receiving  invalid  information  relative  to  those  whose  partners 
were  truthful.  Team  members  discussed  four  topics  intended  to  promote  personal 
communication.  Communication  took  place  FtF  or  via  one  of  three  CMC  modalities:  text,  audio, 
or  audiovisual  (AV).  The  same  interactivity  and  trust  measures  were  used  as  in  Experiment  1 ,  to 
which  were  added  assessments  of  team  members’  truthfulness  and  expertise.  Results  showed 
that  AV  teams  felt  more  connection  and  similarity  and  perceived  team  members  as  more  truthful 
than  did  teams  using  audio  or  text  communication.  Apparently  the  presence  of  visual  nonverbal 
cues  contributed  to  mutuality  with  AV  communication.  As  predicted,  greater  task  load 
(introducing  invalid  information)  reduced  involvement,  mutuality,  and  perceived  truthfulness 
and  expertise.  The  combination  of  task  load  and  communication  modality  altered  these 
relationships.  Under  low  load  (no  deceit  present),  involvement  was  highest  under  audio  and  FtF 
communication  (similar  to  the  previous  experiment).  Under  high  load  (deception  present), 
involvement  was  much  lower  overall,  especially  under  audio  and  text,  but  remained  highest 
under  AV.  Trust  showed  the  same  pattern:  highest  trust  under  low  load  with  the  audio  modality 
but  lowest  trust  under  high  load  with  the  same  modality.  Given  that  in  the  presence  of  invalid  or 
deceitful  information,  team  members  should  be  least  trusting,  the  implication  is  that  the  audio 
modality  is  the  most  sensitive  to  such  differences  and  the  best  modality  to  use  when  there  is 
some  reason  to  doubt  team  members’  truthfulness,  motivations,  or  information.  The  second  best 
modality  for  making  fine  discriminations  was  text.  The  least  effective  were  AV  and  FtF, 
speculatively  due  to  over-reliance  on  misleading  nonverbal  visual  information  available  in  those 
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modalities.  Overall,  these  results  confirm  that  task  load  and  modality  together  influence  the 
communication  process  in  complex  ways. 

The  third  experiment  utilized  our  own  revised  version  of  the  familiar  Desert  Survival 
Problem,  in  which  team  members  must  rank-order  salvageable  items  from  their  overturned  jeep 
in  terms  of  survival  value.  Pairs  interacted  FtF  or  via  text,  audio  or  AV.  One  member  in  half  of 
the  teams  was  asked  to  provide  bogus  arguments  and  evidence  contrary  to  expert 
recommendations,  which  both  team  members  read  beforehand  in  the  form  of  encyclopedic 
information  on  desert  survival.  In  this  experiment,  modality  did  not  directly  affect  the 
communication  process  but  task  load  did:  Higher  load  (presence  of  deceptive  information) 
reduced  involvement  and  mutuality,  trust,  other  credibility  assessments,  and  decision  quality. 
The  implications  are  that  communication  processes  “register”  invalid  information,  even  if  at  an 
unconscious  level,  by  producing  poorer  communication  processes,  which  in  turn  are  correlated 
with  loss  of  trust,  reduced  credibility,  and  poorer  decisions.  A  further,  follow-up  analysis  of  the 
data  from  these  last  two  experiments  revealed  that  FtF  communication  is  still  the  most  expected 
and  positively  regarded  modality  for  communication,  in  part  because  achieving  mutuality  and 
smooth,  productive  interaction  is  easier  with  FtF  communication.  However,  the  CMC  conditions 
are  also  well  regarded  when  task  load  is  low.  The  implication  is  that  if  leaders  and  team 
members  can  manage  the  communication  process  itself  to  be  smooth,  coordinated,  and  useful 
and  to  keep  involvement  up,  then  any  CMC  modality  can  be  used  effectively  under  routine  (low 
load)  conditions. 

The  fourth  experiment  examined  task  load  by  manipulating  the  complexity  of 
information  that  team  members  had  to  digest  before  completing  the  DSP  task.  The  low  load 
version  of  “Surviving  in  the  Desert”  was  the  same  “encyclopedic”  version  used  in  the 
Experiment  3,  which  had  a  12th  grade  readability  level  and  moderate  level  of  linguistic 
complexity.  High  load  versions  were  written  using  far  more  technical  jargon  and  details  from  a 
military  field  manual,  with  a  college  level  of  readability  and  more  complex  vocabulary.  Results 
revealed  that  adverse  effects  from  high  task  load  depended  on  what  form  of  communication  was 
used.  Whereas  with  audio  communication  the  high-load  technical  information  created  greater 
effort,  with  FtF  communication  the  high-load  technical  information  made  the  task  seem  less 
effortful  than  the  low-load  version.  Higher  load  also  created  greater  trust,  perhaps  because  the 
team  members  who  used  it  seemed  more  knowledgeable.  Modality  did  not  affect  trust  directly 
but  did  affect  all  communication  measures.  As  expected,  involvement  and  mutuality  were 
lowest  with  text  communication.  Team  members  were  also  judged  as  the  least  sociable, 
competent,  and  dominant  under  text  and  the  most  so  under  FtF  and  AV.  Overall,  then,  text  was 
the  least  desirable  modality  for  conducting  this  decision-making  task,  suggesting  that  leaders 
may  experience  similar  declines  in  interactivity,  high-quality  communication,  trust  and  team 
performance  when  text  is  the  chief  means  of  communicating.  Interestingly,  task  satisfaction  was 
highest  under  the  combination  of  audio  and  low  task  load,  indicating  that  the  audio  modality’s 
benefits  are  greatest  when  it  is  not  used  for  overly  complex  information  transmission. 

The  fifth  experiment  again  examined  task  load  as  a  form  of  time  pressure,  this  time  by 
comparing  synchronous  (real-time)  communication  versus  asynchronous  (delayed  transmission) 
communication.  The  former  used  text  chat  as  the  means  of  communication;  the  latter  used  email. 
Team  members  again  conducted  the  DSP,  either  during  a  single  series  of  same-time  exchanges 
or  over  the  course  of  a  week.  As  expected,  synchronous  interactions  were  judged  higher  on 
involvement,  mutuality,  sociability,  and  composure,  and  generated  higher  truth  bias 
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(overestimating  truthfulness)  than  did  the  asynchronous  condition.  Although  trust  did  not  differ 
between  the  two  conditions,  those  in  the  synchronous  condition  were  also  less  suspicious  and 
less  motivated  to  monitor  their  partner.  These  findings  strongly  support  the  interactivity 
principle — real-time  communication  promoted  greater  more  involvement,  mutuality,  positive 
regard,  and  belief  in  the  other’s  truthfulness.  Simultaneously,  it  sapped  team  members  of  the 
motivation  to  critically  evaluate  incoming  information.  Implications  for  leadership  are  that 
same-time  communication  is  ideal  for  promoting  conditions  of  high  interactivity  and  trust  but 
detrimental  when  the  objective  is  critical  analysis  and  evaluation  of  information. 

The  sixth,  seventh,  and  eighth  experiments  utilized  C3I  game — first  Scudhunt,  then  an 
original  game  we  developed  called  BusterBunker  and  finally  the  most  recent  game  we 
developed,  called  StrikeCom.  The  general  task  entails  3-  or  4-person  teams  controlling  different 
information  assets  that  they  use  to  search  a  hostile  area.  After  sharing  their  respective 
information  returns  from  their  assets,  the  team  develops  a  unified  strike  plan  to  eliminate  enemy 
strongholds.  This  task  was  conducted  under  the  same  four  modalities  as  used  previously.  In 
some  conditions,  one  team  member  introduced  deceptive  information.  In  others,  teams  either 
had  a  designated  leader  or  were  encouraged  to  work  together  and  select  a  leader.  Initial  results 
have  confirmed  that  greater  task  load  (in  the  form  of  invalid  information)  reduces  group 
performance.  In  other  words,  teams  with  bad  information  fail  to  ferret  it  out  and  discount  it. 
Teams  with  emergent  leaders  also  performed  better  than  those  without  or  those  with  designated 
leaders.  Additional  analysis  is  in  progress. 

A  ninth  inves’  .  ation  considered  how  communication  awareness  affects  leadership,  trust 
and  team  performance.  Interactivity  is  an  essential  mechanism  for  achieving  four  types  of 
awareness:  activity  (knowing  what  others  are  doing),  availability  (knowing  when  others  are 
available  to  meet  or  talk),  process  awareness  (knowing  how  various  pieces  of  the  process  fit  into 
the  big  picture),  and  social  awareness  (having  the  individuating  information  to  know  each 
member’s  social  situation).  Effective  leaders  promote  these  forms  of  awareness,  along  with 
participant  involvement  and  mutuality,  by  structuring  tasks  and  communication  modalities  to 
achieve  them  and  by  engaging  in  behaviors  that  signal  trust,  respect,  and  concern  for  others. 
Achieving  these  objectives  becomes  more  complicated  with  distributed  communication.  Project 
teams  that  were  geographically  dispersed  worked  interdependently  on  a  complex  task  (high  task 
load)  using  electronic  communication  (and  limited  FtF)  over  the  36-day  project  period.  Graduate 
students  served  as  team  leaders.  Results  confirmed  that  greater  awareness  was  a  significant 
predictor  of  team  performance.  High-performing  teams  communicated  more  frequently  (i.e., 
demonstrated  higher  interactivity).  Leaders  who  encouraged  more  daily  contacts  and  more 
awareness  among  team  members  were  more  successful. 

A  tenth  investigation  considered  whether  designated  leadership  would  differ  from  team- 
selected  (i.e.,  emergent)  leadership  and  whether  communication  modality  (FtF,  web 
conferencing,  or  no  interaction)  would  affect  performance.  Results  showed  that  emergent 
leaders  created  more  awareness  than  assigned  leaders.  As  with  the  prior  experiments,  FtF 
interaction  created  greater  mutuality  (“we-ness”)  than  did  the  web  or  no  interaction  condition, 
but  performance  was  actually  highest  in  the  web  condition  with  the  emergent  leader.  This 
suggests  that  FtF  may  be  preferable  for  social  objectives  such  as  creating  group  solidarity  and 
morale  but  that  CMC  may  actually  be  preferable  for  task  focus  if  the  team  has  an  emergent 
(rather  than  assigned)  leader  structuring  the  interaction.  Absent  that,  the  web  format  had  the 
worst  performance,  worse  than  teams  that  did  not  interact.  Other  analyses  are  pending. 
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Final  Summary 


Work  teams  that  communicate  through  new  computer  technologies  fare  no  worse  (or 
perhaps  no  better)  than  teams  that  hold  face-to-face  meetings.  It  all  depends  on  what  the  task  is 
and  what  the  team’s  objectives  are.  Our  series  of  studies  focusing  primarily  on  voice-only, 
audio-visual,  and  text  interfaces,  and  on  several  collaborative  tasks,  found  that  the  degree  of 
interactivity  that  team  members  achieve  affects  team  trust  and  performance,  irrespective  of  the 
type  or  difficulty  of  the  task.  Characteristics  of  interactivity  include  how  connected  participants 
feel  toward  each  other,  how  involved  they  are,  and  the  extent  to  which  they  perceive  each  other 
having  the  same  goals.  Collaborations  that  are  perceived  as  highly  interactive  reflect  favorably 
upon  perceptions  of  the  participants  and  are  associated  with  other  positive  communication 
qualities  such  as  high  quantity  of  information,  efficiency,  critical  analysis,  and  smooth,  relaxed 
and  pleasant  interaction.  These  qualities  in  turn  contribute  to  strong  team  performance.  Some 
technologies  make  high  interactivity  more  likely  than  others  and  result  in  higher  team  trust  and 
performance.  For  example,  same-time  voice  communication  creates  mutual  feelings  of 
connection,  understanding,  and  involvement;  leads  to  richer  and  more  fruitful  discussions,  builds 
trust,  and  results  in  better  decision-making.  More  cognitively  challenging  tasks  may  benefit 
from  the  availability  of  visual  as  well  as  auditory  channels  to  clarify  meanings,  signal 
understanding,  and  exchange  feedback.  But  even  interfaces  that  are  less  “interactive”  (e.g., 
email)  can  still  be  perceived  as  fairly  interactive,  under  the  right  conditions  and  with  the  right 
guidance  from  leaders. 

Not  all  tasks  profit  from  interactivity,  especially  when  longer,  more  thoughtful 
deliberation  is  needed,  or  potentially  invalid  information  is  being  shared  that  needs  greater 
scrutiny.  In  these  cases,  less  interactivity — greater  detachment  and  sense  of  separation — will 
lower  trust  but  may  result  in  better  decision-making.  Real-time  technologies  such  as  instant 
messaging,  cell  phones  and  voice  communication  may  actually  create  mindless  information 
processing  and  hasty  decisions  as  compared  to  older  technologies  such  as  different-time  bulletin 
boards  and  text  exchanges. 

Our  research  did  reveal,  somewhat  unexpectedly,  that  the  degree  of  mental  or  physical 
effort  required  for  a  task  has  variable  effects  on  communication,  trust  and  performance.  People 
are  often  able  to  compensate  for  task  difficulties,  and,  for  some  aspects  of  communication, 
moderate  difficulty  is  preferable  to  low  difficulty,  probably  because  it  keeps  people  more  alert 
and  caught  up  in  the  task.  But  some  interfaces,  such  as  text,  which  requires  typing  rather  than 
speaking,  require  more  mental  effort  to  use  and  therefore  may  be  best  matched  with  less  “taxing” 
tasks.  With  difficult  tasks  such  as  comprehending  more  complex  information,  text-based 
interfaces  also  seem  less  interactive,  making  them  inadvisable  choices  if  trust  and  group  morale 
are  at  stake.  Face-to-face  and  visual  communication,  however,  are  not  the  best  choices  if  the  task 
entails  recognizing  and  assessing  invalid  information. 

Because  the  United  States  Army  funded  this  research,  there  is  considerable  interest  in 
how  these  results  are  informative  for  military  applications.  The  answer  depends  to  some  extent 
on  which  outcomes  are  desirable.  Imagine  that  one  wanted  incoming  information  to  be 
monitored  for  its  veracity — in  such  cases,  audio  communication  seems  to  accomplish  this,  as 
does  bulletin  board  style  communication.  On  the  other  hand,  if  one  desired  simple  information 
exchange  (e.g.,  orders  or  commands  that  should  not  be  questioned),  then  chat-style  text  might  fit 
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the  bill.  In  the  end,  the  complexity  of  communication  is  affected  by  the  technology  that  teams 
use.  It  is  critical  that  technologies  be  fitted  to  desired  aims  and  outcomes  and  that  leaders  either 
select  appropriate  technologies  or  take  measures  to  offset  any  downside  risks  of  using  a  given 
technology.  Informal  leadership  that  emerges  within  a  team,  rather  than  formal  leadership,  may 
have  the  most  influence  on  the  resultant  communication  process  and  outcomes.  Designated 
leaders  are  well  advised  to  monitor  such  interactions  and  to  encourage  frequent  interactions 
among  distributed  team  members. 
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The  Leadership  Formula:  P  x  M  x  D 


Contract  #:  DASW01-01-K-0004  Contract  Dates:  09/01/01-03/01/04 

Institution:  The  Center  for  Outstanding  PI:  Dr.  Reuven  Gal 

Leadership  Co-PI:  Dr.  Micha  Popper 

Research  Summary 

The  present  study  examines  leadership  development  processes  through  a  field  study  in 
the  Israeli  Defense  Force  (IDF).  The  conceptual  framework  suggests  that  leadership  is  a 
function  of  a  given  potential  (P),  a  relevant  motivation  (M)  and  developmental  processes  (D). 
These  three  components  comprise  the  “formula”  (P  x  M  x  D)  for  being  a  leader.  This  “formula” 
and  the  theoretical  framework  from  which  the  formula  is  derived,  expand  the  view  on 
leadership,  by  emphasizing  internal  aspects,  including  personality  predispositions,  motivational 
needs  and  developmental  processes,  that  give  rise  to  leadership  emergence  and  functioning.  Our 
argument  is  that  only  those  who  have  the  potential  represented  by  certain  “ego  resources”  (such 
as  high  self-efficacy  and  low  level  of  anxiety),  who  have  a  strong  motivation  to  lead,  and  who 
are  capable  of  personal  development  can  become  leaders. 

Guided  by  this  conceptual  framework,  we  have  three  main  research  hypotheses.  First, 
‘leaders’  will  have  higher  potential  (“ego”  resources)  compared  to  ‘non-leaders’.  Second, 
‘leaders’  will  have  higher  motivation  to  lead  than  ‘non-leaders’.  Third,  ‘leaders’  will  report 
more  ‘leadership-shaping  experiences’  (LSE)  to  have  occurred  in  their  early  childhood  and 
adolescence  periods,  than  ‘non-leaders’  and  they  will  be  characterized  by  higher  levels  of 
“openness”  to  new  experiences  than  ‘non-leaders’. 

This  study  is  based  on  a  comparative  examination  of  several  samples  of  IDF  soldiers 
(from  Infantry  and  Armour  units)  undergoing  basic-training.  By  distinguishing  ‘leaders’  from 
‘non-leaders’  and  subsequently  assessing  various  P,  M  and  D  measurements,  this  field  study  will 
enable  the  examination  of  our  three  categories  of  hypotheses  related  to  the  leadership  formula: 
potential,  motivation  and  development. 

Contributions  to  Basic  Science 

The  present  study  contributes  to  basic  research  in  the  area  of  leadership  development  by 
addressing  conceptually  and  empirically  three  leadership  concepts:  leadership  potential, 
motivation  to  lead  and  leadership  development.  These  leadership  concepts  are  tested  empirically 
in  our  study  by  using  existing  psychological  instruments  (to  test  the  P  component)  and  by 
constructing  and  forming  new  instruments  (to  test  the  M  and  D  components).  We  have 
expanded  the  “motivation-to-lead”  concept,  a  concept  scarcely  examined  in  the  literature,  both 
substantially  and  technically.  We  have  constructed  three  developmental-leadership  assessment 
tools:  an  openness  to  experience  questionnaire,  a  LSE  interview  and  a  LSE  questionnaire. 
Finally,  we  have  formed  a  questionnaire  identifying  ‘leaders’  and  ‘non-leaders’. 
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Potential  Army/Military  Applications 


This  research  is  conducted  in  the  Israeli  Defense  Force  (IDF).  Unlike  civilian 
organizations,  where  leadership  is  diffused  in  the  manager’s  functions  and  formal  authority,  in 
the  military  ‘net  leadership’  is  much  more  prominent  and,  in  fact,  is  an  essential  driving  force 
that  generates  the  soldiers’  readiness  to  risk  their  lives.  Furthermore,  within  the  military,  the 
distinction  between  ‘leaders’  and  ‘non-leaders’  is  much  more  apparent  than  in  most  other 
institutions.  This  distinction  is  somewhat  contaminated,  however,  in  those  military  organizations 
(such  as  the  U.S  Army),  in  which  the  differentiation  between  officers  and  non-officers  is  done 
‘institutionally’  (i.e.  through  military  academies),  rather  than  ‘naturally’  (namely,  through  a 
gradual  selection  process).  A  unique  example  of  the  latter  type  is  the  IDF:  Starting  on  an  equal 
footing,  all  Israeli  conscripts  undergo  a  continuous  selection  process  during  their  initial  training 
period.  Through  this  prolonged  process  emerge  those  who  eventually  become  officers  or 
NCO’s.  This  unique  situation  makes  the  IDF  an  extraordinary  natural  laboratory  to  examine 
leadership  development.  Moreover,  this  field  study  is  conducted  on  two  different  army  corps, 
Infantry  and  Armor,  in  order  to  examine  whether  they  reflect  different  leadership-forming 
processes. 

Overall,  this  study  may  provide  a  unique  opportunity  to  understand  how  ‘leaders’  differ 
from  ‘non-leaders’,  what  is  it  that  makes  the  ‘net-leadership’,  and  how  it  develops.  Furthermore, 
our  research  may  unravel  some  of  the  mysteries  related  to  the  nature-nurture  origins  of 
leadership,  as  well  as  suggest  some  important  and  practical  implications  related  to  leader 
selection  and  development  (i.e.  training)  in  any  modem  military. 

Accomplishments 

So  far,  we  have  completed  the  first  round  of  questionnaires  and  interviews.  By  February 
2003,  we  obtained  questionnaires  from  402  IDF  soldiers  and  we  performed  60  LSE  interviews. 
We  have  finished  analyzing  the  leadership  potential  (P)  questionnaires  and  the  results  are  very 
promising.  All  potential  variables  (“ego  resources”)  significantly  differentiate  between  ‘leaders’ 
and  ‘non-leaders’.  These  results  were  presented  on  June  17-18,  2003  in  the  IPR  meeting  at  Fort 
Leavenworth.  An  academic  journal  paper,  based  on  these  results,  is  already  in  its  final  version. 

We  now  are  analyzing  the  quantitative  data  concerning  the  Motivation  to  Lead  (MTL) 
element  and  the  Development  element  (D).  The  initial  findings  are,  again,  promising.  There  are 
significant  differences  in  the  motivation  to  lead  between  soldiers  defined  as  ‘leaders’  and  those 
defined  as  ‘non-leaders’.  Also,  as  expected,  soldiers  defined  as  ‘leaders’  are  more  open  to  new 
experiences  than  those  defined  as  ‘non-leaders’. 

The  qualitative  data  concerning  development  issues  is  now  being  analyzed. 

In  May  2003  we  started  the  second  round  of  questionnaire  administration.  In  this  phase 
we  return  to  our  original  subjects,  and  administer  the  three  questionnaires  that  measure  those 
attributes  that  are  conceptually  dynamic  and  may  change  over  time  (General  Self-efficacy, 
Motivation  to  Lead,  and  Openness  to  Experience). 
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Future  Plans 


We  plan  to  finish  the  second  phase  of  questionnaire  administration  by  September  2003 
and  the  follow-up  interviews  by  December  2003.  We  will  continue  analyzing  the  data  and  will 
draw  conclusions  concerning  each  of  our  three  components  (P,  M  &  D)  as  well  as  the 
interrelationships  between  these  components.  Finally,  we  are  seriously  thinking  about 
developing  further  studies  based  on  the  promising  data  we  have  obtained. 


36 


Leadership:  Enhancing  Team  Adaptability  in  Dynamic  Settings 

Contract  #:  1435-04-03-CT-71272  Contract  Dates:  01/24/03-01/23/05 

Institution:  University  of  Maryland  -  College  Park  PI:  Dr.  Katherine  J.  Klein 

Co-PI:  Dr.  Steve  W.  J.  Kozlowski, 

Dr.  YanXiao 


Research  Summary 

The  U.  S.  Army  Vision  is  to  transition  the  Army  into  an  Objective  Force  that  is 
strategically  responsive,  dominant  at  every  point  on  the  spectrum  of  operations,  and  capable  of 
transitioning  across  mission  requirements  without  loss  of  momentum.  To  fulfill  its  missions,  the 
Objective  Force  of  the  Army  must  be  responsive,  deployable,  agile,  versatile,  lethal,  survivable, 
and  sustainable. 

Such  responsiveness,  deployability,  agility,  and  versatility  are  the  hallmarks  of  adaptive 
team  performance  -  the  ability  of  team  members  to  individually  and  cooperatively  apply  their 
knowledge  and  skills  to  the  resolution  of  urgent,  complex,  novel,  and  ambiguous  problems  in 
dynamic  work  settings.  In  such  settings,  rigid  adherence  to  standard  operating  procedures  and 
routines  is  impossible.  Adaptability  is  essential. 

What  then  can  leaders  do  to  develop  and  enhance  the  adaptability  that  is  so  essential  for 
the  Objective  Force?  Existing  theory  and  research  offer  few  explicit  guidelines.  Scholars  reveal 
increasing  awareness  of  the  importance  of  individual  and  team  adaptability  for  the  Army  of  the 
Future  -  indeed,  for  the  workplace  of  the  future,  yet  empirical  studies  of  adaptive  team 
performance  within  dynamic  work  environments  are  virtually  nonexistent. 

To  begin  to  fill  this  gap,  we  have  begun  a  multi-level,  multi-method  research  program 
designed  to  illuminate  the  influence  of  individual  differences,  team  characteristics,  and 
leadership  on  individual  and  team  adaptive  performance  within  a  dynamic  work  environment. 
Our  research  addresses  three  questions: 

•  How  do  individuals  acquire  the  knowledge  and  skills  necessary  for  adaptive  individual 

performance? 

•  How  do  teams  comprised  of  adaptive  individuals  acquire  the  shared  knowledge  and  skills 

necessary  for  adaptive  team  performance? 

•  What  can  leaders  do  to  enhance  these  processes? 

Contributions  to  Basic  Science 

Our  research  will  contribute  to  basic  science  in  four  key  respects.  First,  our  research  will 
shed  new  light  on  team  leadership.  Existing  models  of  leadership  focus  on  the  effects  of 
leadership  on  individual  employees  or  on  the  organization  as  a  whole.  Theory  and  research  on 
team  leadership  -  on  the  ways  in  which  leaders  influence  team  members’  interdependent, 
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coordinated  work  performance  -  is  very  limited.  Our  research  promises  to  clarify  the  leader 
traits,  behaviors,  and  strategies  that  enhance  team  performance. 

Second,  our  work  will  extend  research  and  theory  on  how  individuals  operating  in 
complex  and  dynamic  team  settings  learn  in  ways  that  contribute  to  both  individual  and  team 
effectiveness.  By  better  understanding  this  process  of  learning  and  adaptation,  we  will  be  able  to 
show  how  leaders  can  develop  more  adaptive  and  effective  teams. 

Third,  by  examining  individual  and  team  learning  and  adaptation  over  time,  we  will 
contribute  to  multilevel  theory  and  research  methods,  clarifying  appropriate  strategies  for 
multilevel  theory-building  and  research  on  processes  occurring  over  time.  Such  longitudinal 
processes  have  relatively  rarely  been  considered  in  multilevel  theory  and  research. 

Finally,  by  employing  qualitative  and  quantitative  field  research  methods  as  well  as 
experimental  laboratory  research,  our  research  will  provide  an  example  of  the  use  of 
complementary  research  strategies  to  strengthen  and  cross-validate  research  conclusions. 

Potential  Army/Military  Applications 

Our  research  will  identify  theoretically  driven,  empirically  grounded,  and  experimentally 
validated  antecedents  of  individual  adaptability  and  team  adaptability.  In  addition,  our  research 
will  yield  specification  of  leadership  behaviors,  capabilities,  and  skills  that  critically  influence 
the  development  of  individual  and  team  adaptability.  We  anticipate  that  our  findings  may  be  put 
to  use  in  selecting  individual  soldiers,  teams,  and  leaders  most  likely  to  perform  adaptively  in 
dynamic  settings.  Further,  our  findings  may  suggest  training  interventions  and  intended  training 
outcomes  to  enhance  the  adaptive  performance  of  individuals,  teams,  and  leaders. 

Accomplishments 

The  first  phase  of  our  research  program  is  now  underway,  with  one  focus  of  activity 
centered  on  the  Trauma  Resuscitation  Unit  of  the  Baltimore  Shock  Trauma  Center  and  the  other 
focus  of  activity  centered  on  the  Adaptive  Performance  Training  Lab  at  Michigan  State 
University. 

We  are  in  the  early  stages  of  qualitative  research  in  the  Trauma  Resuscitation  Unit  of  the 
Baltimore  Shock  Trauma  Center.  There,  we  have  conducted  background  interviews  with  nurses 
and  attending  surgeons  regarding  the  resident  program  that  is  the  focus  of  our  research  at  the 
Shock  Trauma  Center.  These  background  interviews  provide  the  groundwork  for  more  intensive 
qualitative  data  collection  in  the  coming  months,  as  described  below.  In  preparation  for  further 
field  data  collection,  we  have  reviewed  and  analyzed  background  documents  regarding  the 
Shock  Trauma  Center  as  well  as  interview  data  collected  in  our  prior  research,  but  not  previously 
analyzed  in  depth. 

Initial  work  at  the  Adaptive  Performance  Training  Lab  has  involved  an  examination  of 
the  scientific  research  literatures  that  pertain  to  team  leadership  functions,  team  development 
processes,  and  performance  adaptability.  We  are  in  the  process  of  abstracting  and  compiling  that 
work;  distilling  core  team  leadership  functions;  and  synthesizing  new  team  leader  functions  that 
promote  individual  learning,  team  development,  and  adaptive  performance  capabilities.  The 
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research  model  that  will  result  from  this  effort  will  be  used  to  specify  the  core  constructs,  key 
processes,  and  primary  outcomes  that  will  be  examined  in  subsequent  lab  experiments. 

Future  Plans 

In  the  next  four  months,  we  will  conclude  our  qualitative  study  of  individual  and  team 
adaptive  performance  in  the  Trauma  Resuscitation  Unit  of  the  Shock  Trauma  Center.  Our 
qualitative  research  is  designed  to  ground  our  subsequent  quantitative  examinations  of  leadership 
and  individual  and  team  adaptability  in  a  real-life  dynamic  setting.  On  the  basis  of  our 
qualitative  findings,  we  will:  (a)  develop  additional  hypotheses  regarding  the  antecedents  of 
individual  and  team  knowledge  acquisition  and  adaptive  performance;  and  (b)  refine  and  develop 
survey  and  other  measures  that  will  allow  us  to  test  our  individual-  and  team-level  hypotheses 
using  a  quantitative,  longitudinal,  survey  methodology  in  the  Shock  Trauma  Center. 

During  the  remainder  of  the  upcoming  year,  we  will  conduct  longitudinal  survey  research 
to  test  our  hypotheses  regarding  the  antecedents  of  individual  and  team  adaptive  performance. 
We  plan  to  gather  data  from  individual  residents  and  resident  teams  (that  is,  survey  data 
regarding  individual  residents  and  regarding  each  team  of  residents  as  a  whole)  at  least  two  times 
each  month  (early  in  the  life  of  the  team  and  late  in  the  life  of  the  team).  Our  goal  is  to  allow 
longitudinal  assessment  of  the  development  of  individual  knowledge  and  skills  and  of  the 
development  of  shared  team  knowledge. 

Development  of  the  research  model,  specification  of  primary  experimental  factors,  and 
operationalization  of  measures  is  expected  to  be  completed  over  the  next  four  months.  We  will 
then  initiate  lab  research  that  examines  antecedents  of  individual  learning  and  the  development 
of  adaptive  capabilities. 
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Learning  Leadership  Skills  in  Distributed  Training  Scenarios: 
Diagnosing  Strategies  in  Scenarios  Using  Latent  Semantic  Analysis 

Contract  #:  DASW01-00-K-0017  Contract  Dates:  9/1/00-8/31/03 

Institution:  George  Mason  Univ/Univ  of  Pittsburgh  PI(s):  Christian  Schunn 

Research  Summary 

The  primary  objectives  of  this  research  are  to  (a)  test  a  theoretical  model  of  expertise  in 
complex  tacit  knowledge  domains  like  leadership  and  (b)  test  a  diagnosis  and  training  approach 
that  is  based  on  that  theoretical  model  of  expertise. 

The  theoretical  model  being  applied  to  leadership  assumes  that  one  important  component 
of  expertise  is  the  effective  and  appropriate  use  of  strategies.  Expert  strategy  use  generally 
differs  from  novice  strategy  use  along  five  dimensions  (Lemaire  &  Siegler,  1995;  Schunn  & 
Reder,  1998).  In  order  to  provide  appropriate  training  to  a  novice,  one  has  to  diagnose  the 
particular  type  of  strategy  use  problems  exhibited  by  the  given  novice.  The  particular  problems 
include  (a)  missing  strategies  entirely,  (b)  under-using  generally  effective  strategies,  (c)  poorly 
executing  strategies,  and  (d)  executing  good  strategies  at  the  wrong  times,  (e)  failing  to  adapt 
strategies  to  a  changing  environment. 

Diagnosing  strategy  use  in  complex  domains  like  leadership  presents  a  difficult 
challenge.  Thus,  the  secondary  objectives  of  this  research  are  to  explore  mechanisms  for 
automatically  diagnosing  strategy  use  and  larger  scale  problems  in  patterns  of  strategy  use  (along 
the  five  described  dimensions)  in  complex  domains,  particularly  the  strategies  found  in  written 
responses.  If  such  a  method  can  be  developed,  then  this  strategies-training  approach  can  be 
effectively  applied  in  distributed,  distance-training  situations. 

Contributions  to  Basic  Science 

The  current  work  advances  both  our  knowledge  of  the  psychology  undering  expertise  and 
computer  science  of  intelligent  tutoring.  The  model  of  expertise  that  I  am  testing  has  only  been 
demonstrated  in  very  simple  domains  (like  child  mathematics  learning),  and  it  is  unclear  whether 
models  of  expertise  from  very  simple  domains  will  be  applicable  to  expertise  in  complex 
domains  like  platoon  leadership.  Second,  intelligent  tutoring  systems  have  traditionally  limited 
the  ways  in  which  the  person  could  interact  with  the  systems.  Recently,  approaches  have 
attempted  to  include  more  natural  interactions,  like  spoken  and  written  dialog  interaction. 
However,  much  basic  science  needs  to  be  done  in  developing  methods  for  parsing  spoken  or 
written  dialog.  One  such  open  problem  is  diagnosing  strategies  and  the  proposed  work  tests  a 
very  new  scheme  for  solving  this  problem. 

Potential  Army/Military  Applications 

The  application  of  our  intelligent  tutoring  work  is  clear:  if  the  automatic  strategy 
identification  methodology  works,  we  can  build  a  successful  tutoring  system.  By  using  it,  then, 
we  will  have  developed  a  powerful  methodology  for  intelligent  tutoring  in  many  complex,  ill- 
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defined  domains  which  are  found  in  many  military  settings,  including  the  platoon  leadership 
setting  being  studied  directly. 

We  also  expect  to  learn  more  about  what  common  strategies  can  be  effectively  applied 
across  a  wide  range  of  platoon  leadership  situations.  This  knowledge  in  itself  could  also  be 
applied  to  other,  more  traditional  training  methods  beyond  the  particular  automated  method 
being  pursued  here. 


Accomplishments 

To  develop  a  base  set  of  diagnostic  and  training  examples,  we  have  expanded  the  original 
set  of  15  platoon  leadership  scenarios  in  the  TKML  (Tacit  Knowledge  Military  Leadership)  to  50 
scenarios.  These  50  scenarios  cover  the  main  types  of  skills  typically  associated  with  platoon 
leadership.  The  new  scenarios  were  adapted  from  scenarios  presented  in  the  Army  Leadership 
Tacit  Knowledge  Corpus.  Because  these  scenarios  are  to  be  used  for  written  responses  and  our 
goal  is  to  diagnose  particular  leadership  strategies  in  the  responses,  the  scenarios  were  designed 
to  require  a  single  strategy  response  rather  than  a  long  list  of  different  strategies. 

In  preparation  for  the  LSA  phase  of  the  project,  we  acquired  the  specialized  computer 
and  software  to  run  LSA  and  successfully  conducted  basic  test  runs  with  LSA.  We  then  tested 
LSA  using  various  military  leadership  databases  as  input,  including  the  vignettes  from  the  Army 
Leadership  Tacit  Knowledge  Corpus,  the  Army  Military  Leadership  Field  Manual  FM  22-100, 
and  numerous  text  documents  related  to  military  leadership  found  on  the  web  (many  of  which 
can  be  found  at  http://www.au.af.mil/au/awc/awcgate/awc-ldr.htm)  producing  a  total  text  corpus 
of  0.5  million  words. 

We  collected  data  from  25  ROTC  cadets,  with  each  cadet  generating  text  responses  to  at 
least  10  open-ended  scenarios.  We  developed  a  set  of  13  leadership  strategies  (e..g,  planned 
action,  authoritarian,  avoidance,  punishing,  seek  additional  information)  based  on  cards  sorts  of 
possible  options  in  the  Platoon  TKML  ratings  task,  and  then  conducted  some  testing  of  these 
strategies  on  the  open-ended  text  data. 

To  develop  an  automatic  method  for  coding  strategies  in  the  open-ended  text  responses  to 
be  used  in  the  intelligent  tutoring  system,  we  went  through  a  number  of  steps.  First,  a  research 
assistant  generated  an  example  of  how  to  implement  each  of  the  13  strategies  in  each  of  10 
scenarios  (selecting  only  10  as  a  trial  set).  These  implementations  were  called  scenario-strategy- 
ideals.  Then  LSA  compared  each  open-ended  response  to  these  scenario-strategy-ideals  and 
assigned  the  response  to  the  strategy  category  with  the  best  matching  ideal.  This  method  proved 
to  be  better  than  chance,  but  well  below  human  coding  accuracy  and  well  below  usable  levels  for 
later  use  in  intelligent  tutoring. 

Through  a  gradual  process  of  refinement  in  coding  actual  open-ended  responses  using 
these  13  strategies,  we  narrowed  the  set  down  to  10  general  strategies  that  could  be  coded  more 
reliably.  We  then  asked  a  different  set  of  28  ROTC  cadets  to  generate  implementatons  of  each  of 
the  strategies  in  each  of  the  scenarios  (5  strategies  for  5  scenarios  for  each  given  cadet).  Because 
there  appeared  to  be  multiple  legitimate  ways  to  implement  a  given  strategy  in  a  given  scenario, 
we  wanted  data  on  the  variety  of  ways  this  could  be  achieved.  Coders  carefully  went  through 
this  forced  strategy  data  to  make  sure  that  the  cadets  did  indeed  respond  with  the  given  strategy 
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in  the  given  scenerio  context.  Only  a  small  proportion  of  the  forced  data  was  removed  due  to  not 
implementing  the  indicated  strategy. 

With  the  forced  strategy  data  in  hand,  we  then  reconducted  the  LSA  analyses.  This  time 
the  LSA  strategy  coding  results  were  better  but  still  below  human  coding  accuracy — 
approximately  50%  of  human  performance.  Exploration  found  that  the  richness  of  the  text  input 
did  not  matter,  nor  did  many  variations  of  the  matching  algorithm.  It  is  our  current  belief  that 
approaches  like  LSA  which  do  not  use  structural  information  of  the  text  are  not  well  suited  for 
strategy  classification  in  complex  domains.  We  built  a  prototype  of  the  tutoring  system  (using  a 
web-based  linux  system  using  mySQL  and  PhP).  However,  without  accurate  automatic  strategy 
classification,  we  chose  not  to  test  the  system  in  a  training  situation. 

To  study  the  effects  of  expertise  on  these  strategies,  we  also  collected  data  on  the  open- 
ended  task  from  undergraduates  completely  unfamiliar  with  either  leadership  or  the  Army.  We 
also  collected  comparable  data  from  60  experts  (Its,  capts,  majors)  last  summer  from  two  military 
bases  as  part  of  Umbrella  week  activities.  We  found  that  expertise  in  leadership  does  consist  of 
having  better  strategies,  selected  with  better  base-rates,  and  selected  for  particular  problems  in  a 
more  appropriate  fashion. 

To  study  the  issue  of  quality  of  strategy  implementation,  we  selected  responses  from  each 
of  the  groups  and  gave  them  to  a  new  group  of  experts  (this  time  25  Its,  capts,  majors  from  one 
base  during  Umbrella  week),  and  asked  them  to  assess  how  good  a  job  the  person  did  of 
implementing  the  given  strategy  (as  we  coded  it)  regardless  of  whether  that  was  the  best  strategy 
to  use  in  the  given  scenario.  Thus  far,  we  have  not  seen  noticeable  differences  in  the  quality  of 
implementation,  but  we  have  only  begun  to  analyze  this  data. 

We  have  also  conducted  several  basic  research  studies  to  examine  the  ways  in  which 
adapting  strategy  use  to  changing  environments  plays  a  role  in  expert  performance.  In  the  first 
study,  we  found  that  there  are  individual  differences  in  how  well  people  adapt  their  strategy  use 
to  changes  in  how  well  the  strategies  worked  (using  a  simple  problem  solving  task  in  which  we 
had  perfect  control  over  strategy  effectiveness).  In  the  second  study,  we  showed  that  the 
individual  differences  were  related  to  how  explicitly  aware  people  were  of  the  changes  in  the 
environment,  but  that  unaware  people  did  adapt  to  some  (lesser)  extent.  We  also  showed 
(surprisingly)  that  working  memory  was  not  related  to  adaptivity.  In  the  third  study,  we  showed 
that  adaptivity  level  was  not  related  to  motivational  levels,  neither  intrinsic  (as  measured  by  self- 
report)  nor  manipulated  (performance  paid  or  unpaid).  In  the  fourth  study,  we  found  that  these 
individual  differences  were  stable  within  different  measures  of  adaptivity  in  a  given  task,  and 
across  two  different  tasks  (a  simple  problem  solving  task  and  a  3-d  navigation  task).  In  the  fifth 
study,  we  found  that  adaptivity  levels  were  uninfluenced  by  the  insertion  of  other  adaptivity 
learning  tasks  or  filler  time,  consistent  with  an  implicit  model  of  learning  strategy  preferences. 
Overall,  we  have  an  interesting  juxtaposition  of  strategy  choice  learning  which  appears  to  be 
influenced  by  explicit  awareness,  but  not  influenced  by  factors  typically  associated  with  explicit 
learning  mechanisms  (e.g.,  interference,  working  memory). 

Final  Summary 

Overall,  we  partially  validated  our  model  of  expertise.  We  found  that  expertise  in 
platoon  leadership  consists  of  having  better  strategies,  understanding  which  strategies  are  better 
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overall,  and  knowing  when  to  use  those  strategies.  Contrary  to  expectations,  we  did  not  find  that 
experts  implemented  the  same  strategies  more  effectively,  but  we  are  still  exploring  this  issue. 

We  have  also  elaborated  our  understanding  of  how  people  adapt  their  strategy  use  to 
changing  environments.  We  found  out  that  there  appears  to  be  stable  individual  differences  in 
how  well  people  adapt  their  strategy  selections  to  changing  environments.  We  have  also 
produced  a  puzzle  for  what  underlies  this  strategy  adaptivity.  We  learned  that  it  has  both 
implicit  and  explicit  components  and  that  the  explicit  component  is  required  for  rapid  adaptivity. 
However,  we  found  that  working  memory  capacity  and  memory  interference  factors,  which  we 
expected  to  influence  adaptivity,  do  in  fact  not  influence  adaptivity. 

At  the  level  of  intelligent  tutoring,  we  explored  one  approach  to  automatic  identification 
of  strategies  and  found  that  it  was  able  to  make  some  progress  on  the  problem,  it  was  not  likely 
to  reach  practically-useful  levels.  Since  this  kind  of  approach  is  still  in  its  rapid-growth  phase  in 
the  research  community,  failures  like  the  current  one  in  applicability  of  the  approach  are  likely  to 
be  very  informative  to  other  researchers  in  the  area. 

We  did  develop  a  methodology  for  identifying  higher-level  problems  in  strategy  use 
(related  to  our  different  dimensions  of  strategy  expertise),  and  this  methodology  did  prove  useful 
and  will  feed  into  future  intelligent  tutoring  work. 
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Developing  Effective  Military  Leaders: 

Facilitating  the  Acquisition  of  Experience-Based,  Tacit  Knowledge 


Contract  #:  DASW01-00-K-0014  Contract  Dates:  9/01/00  -  9/30/04 

Institution:  Yale  University  PI:  Robert  J.  Sternberg 

Research  Summary 

Army  officers  are  facing  a  leadership  environment  that  is  more  diverse  and  more  rapidly 
changing  than  ever.  This  diversity  and  change  stems  from  an  increased  variety  in  the  nature  of 
missions  that  Army  personnel  must  undertake,  an  ever-widening  array  of  sophisticated 
technology  used  to  conduct  these  missions,  and  shared  decision-making  capability  across  ranks 
when  mission  objectives  are  rapidly  modified. 

This  requires  that  officers  leam  from  their  experiences  quickly  and  efficiently.  The 
primary  goal  of  the  “developing  effective  military  leaders”  project,  therefore,  is  to  explore  ways 
to  improve  people’s  ability  to  acquire  knowledge  from  their  experiences  and  to  use  this 
knowledge  to  solve  practical  problems.  We  have  constructed  three  different  sets  of  thinking 
exercises,  all  of  which  are  based  on  Robert  J.  Sternberg’s  theory  of  practical  intelligence  and 
tacit  knowledge,  in  order  to  identify  a  “best  practice”  for  enhancing  experience-based  learning. 
Each  intervention  targets  a  different  aspect  of  tacit  knowledge:  (a)  the  conditions  that  must  be 
met  to  make  optimal  practical  decisions;  (b)  the  actions  that  best  correspond  to  particular 
conditions;  or  (c)  both.  This  work  represents  an  extension  of  previous  research  indicating  a 
relation  between  experience-based  (or  tacit)  knowledge  and  occupational  performance  in  that  we 
are  attempting  (a)  to  better  characterize  the  cognitive  processes  that  occur  while  people  use  tacit 
knowledge;  and  (b)  to  understand  how  these  processes  may  be  facilitated  through  training. 

Contributions  to  Basic  Research 

A  theory  that  attempts  to  explain  human  functioning  in  everyday  life  is  by  necessity 
complex  and  multifaceted.  Testing  and  improving  this  theory  is  a  long  process,  in  which  the 
theory  is  addressed  piece  by  piece.  The  major  contribution  of  the  current  work  to  basic  science 
is  the  illumination  of  a  previously  unexplored  piece  of  Sternberg’s  triarchic  theory  of 
intelligence.  The  piece  we  are  exploring  outlines  the  cognitive  processes  involved  in  acquiring 
tacit  knowledge  from  experience,  the  problem-solving  skills  involved  in  solving  complex 
practical  problems,  and  the  interaction  between  knowledge  and  problem  solving  in  expert 
performance.  We  will  test  whether  a  set  of  instructional  thinking  exercises  based  on  this  piece  of 
the  triarchic  theory  can  “speed  up”  the  experience-based  learning  process  and  improve  practical 
problem  solving.  We  expect  the  results  from  this  test  to  lead  to  additional  refinement  of  the 
triarchic  theory  and  contribute  to  a  growing  knowledge  base  with  regard  to  expertise  and 
leadership  development. 
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Potential  Army/Military  Applications 


Because  the  complex  and  ever-changing  nature  of  job  environments  precludes  teaching 
leaders  what  must  be  done  in  every  problem  situation,  leadership  instruction  must  instead 
provide  leaders  with  the  skills  to  use  experience  as  their  career-long  mentor  and  source  of 
feedback.  We  expect  our  research  findings  to  have  specific  implications  for  the  design  of 
leadership  development  course  materials  delivered  via  distributed-learning  methods.  We  also 
expect  that  our  research  findings  will  provide  an  effective  target  for  more  intensive  classroom- 
based  leadership  development  efforts. 


Accomplishments 

Our  accomplishments  over  the  past  year  involve  instrument  development,  data  collection, 
and  preliminary  data  analysis.  First,  over  the  summer  of  2002,  we  developed  two  new  case  study 
scenarios  to  assess  platoon-level  tacit  knowledge.  The  scenarios  were  designed  based  on  tacit- 
knowledge  theory  with  the  help  of  military  personnel  (LTC  Mike  Prevou  and  LTC  (Ret.)  Chuck 
Powell)  to  make  them  more  meaningful  for  Army  officers  than  previous  scenarios.  Also  during 
this  time  we  audited  a  three-day  segment  of  the  Tactical  Command  Development  Course  at  Ft. 
Leavenworth  in  order  to  fine-tune  the  thinking  exercises  in  our  instructional  interventions, 
making  them  more  compatible  with  the  kind  of  practical-problem  solving  exercises  conducted 
during  existing  command  training. 

Second,  in  August  of  2002,  we  conducted  a  pilot  study  with  1st  Infantry,  1st  Battalion  at 
West  Point  to  assess  the  fidelity  of  our  case  study  scenarios  and  the  feasibility  of  our 
instructional  interventions.  During  this  study  we  worked  with  23  non-commissioned  officers, 
who  completed  the  interventions  and  rated  the  quality  of  the  newly  developed  scenarios.  Our 
discussions  with  the  NCOs  led  to  important  revisions  of  (1)  the  intervention  thinking  exercises  to 
make  them  less  technical  and  more  accessible  to  Army  participants;  (2)  the  problem-solving 
questions  following  the  case  study  scenarios,  again  to  make  them  less  technical  and  more 
accessible  to  Army  participants;  and  (3)  the  paper-and-pencil  format,  changing  it  from  requiring 
written  answers  to  requiring  answers  provided  over  digital  voice  recorders.  Participants  noted 
that  the  writing  demand  in  our  interventions  was  quite  high  and  that  being  able  to  think  aloud 
would  be  more  natural. 

Third,  in  October  of  2002  and  March  of  2003  we  collected  data  at  Fts.  Sill  and  Drum, 
respectively,  working  with  101  officers  (LTs,  CPTs,  MAJs,  and  LTCs)  before  access  to  troop 
support  was  terminated  due  to  deployment  to  Iraq.  Since  that  time,  we  have  worked  to  transcribe 
the  digital  voice  files  created  by  participants,  to  enter  data  from  written  materials,  to  develop  a 
comprehensive  scoring  rubric  for  scoring  open-ended  case  study  reponses,  and  to  begin  analysis 
of  the  data  collected. 

Our  intial  findings  indicate  that  officers  of  all  four  ranks  felt  that  they  learned  something 
from  the  interventions,  that  the  interventions  were  useful  for  understanding  how  to  learn  from 
experience,  that  the  interventions  were  relevant  to  the  leadership  problems  they  typically  face, 
and  that  they  plan  to  use  future  experiences  as  opportunities  for  learning.  These  positive 
attitudes  were  mirrored  by  performance  benefits  experienced  by  officers  in  the  intervention 
conditions.  Overall,  interventions  allowed,  on  average,  officers  to  maintain  performance  from 
pre-test  to  post-test,  whereas  the  performance  of  officers  in  the  control  condition  went  down 
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from  pre-test  to  post-test.  In  addition,  interventions  were  differentially  effective  in  maintaining 
performance,  in  accordance  with  predictions.  These  differential  effects  could  not  be  explained 
by  differences  in  cognitive  ability,  attitudes  towards  leadership  instruction,  or  differences  in  pre¬ 
test  scores.  We  do  not  have  an  explanation  at  this  time  for  why  performance  went  down  in  the 
control  group,  but  we  do  not  have  any  reason  to  suspect  that  control  group  was  different  from  the 
intervention  groups  in  any  systematic  way.  Finally,  captains  gained  the  most  from  participating 
in  an  intervention,  with  lieutenant  colonels/majors  gaining  slightly  less,  and  lieutenants  showing 
a  decrement  in  performance  after  participating.  It  appears  that  a  well-developed  knowledge 
base,  combined  with  a  newly  developing  set  of  reflective  thinking  skills  is  required  to  maximally 
benefit  from  the  interventions. 


Future  Plans 

Our  plans  for  next  year  are  twofold.  First,  we  plan  to  continue  collecting  data  with  Army 
participants,  once  troop  support  is  again  permitted.  We  would  like  to  work  with  an  additional 
140  officers  in  order  to  test  our  hypotheses  with  adequate  statistical  power.  Second,  we  plan  to 
design  an  auxiliary  experiment  to  be  implemented  with  college  students.  In  this  experiment,  we 
will  test  the  same  hypotheses  as  with  the  Army  participants  but  we  will  use  a  more  extensive, 
classroom-based  intervention  with  long-term  follow-up  on  participant  performance  after 
intervention. 
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Distant  Leadership  Under  Stress 

Contract#:  DASW01-99-K-0003  Contract  Dates:  08/01/99-07/31/02 

Institution:  University  of  Maryland,  Baltimore  Pi’s:  Yan  Xiao 

Co-PI’s:  Colin  Mackenzie  &  Katherine 
Klein 


Research  Summary 

The  overall  objective  of  this  project  is  to  investigate  leadership  effectiveness  when 
leaders  are  distanced  from  the  team.  Since  the  role  of  leadership  in  team  performance  is 
important,  understanding  how  leadership  impacts  team  performance  is  important  to  military  as 
well  as  to  civilian  organizations.  With  the  widespread  use  of  electronic  communication 
technologies,  it  has  become  essential  to  establish  a  theoretical  and  empirical  basis  for  predicting 
how  new  communication  technologies  may  impact  leadership  and  team  performance.  The 
specific  aims  of  the  research  program  are:  (a)  to  develop  a  matrix  of  leadership  functions  and 
situations  in  which  specific  functions  are  needed;  (b)  to  develop  a  leadership  model  which 
prescribes  the  nominal  processes  through  which  a  leader  controls  and  influences  team  activities, 
either  co-located  or  at  a  distance;  (c)  to  develop  leadership  measures  applicable  to  leadership  in  a 
dynamic,  team  environment;  and  (d)  to  conduct  a  series  of  prospective  studies  in  a  real,  event- 
driven,  stressful  environment  to  evaluate  the  impact  of  various  communication  modalities  on 
leadership  using  the  measures  developed. 

Contributions  to  Basic  Science 

Our  research  has  several  unique  features  as  compared  to  previous  leadership  research. 
First,  our  empirical  data  is  from  a  real  environment.  Second,  our  study  environment  allows 
direct  observations  and  audio-video  recordings  of  real  team  activities  under  stressful  situations 
and  the  manipulation  of  communication  modalities  between  leaders  and  team  members.  Third, 
the  task  situation  characteristics  (high-risk,  high-tempo)  provide  a  basis  for  studying  highly 
reliable,  rapidly  assembled  and  flexible  teams.  Fourth,  the  empirical  research  triangulates  three 
types  of  data  sources:  interviews,  video  analysis,  and  briefings  with  experiment  subjects. 

This  project  made  two  key  contributions  to  team  leadership  basic  research.  First,  our 
research  established  the  phenomenology  of  team  structures  and  how  the  structures  adapted  to 
various  factors.  Through  analysis  of  communication  structures,  we  examined  changes  in  team 
structures  due  to  the  impact  of  distant  leadership,  team  experience,  and  task  urgency.  We 
proposed  a  set  of  team  structure  archetypes  as  a  way  to  reflect  the  impact  of  team  experience, 
leader  location,  and  task  urgency.  Previous  studies  on  leadership  and  team  performance  have  not 
examined  team  structure  in  this  manner.  Instead,  research  has  typically  assumed  only  two  team 
structures:  1)  undifferentiated  (such  as  self-managed  teams),  or  2)  a  leader- follower/s  dichotomy. 

Second,  the  research  provided  further  evidence  of  dynamic  team  leadership  functions 
among  team  members.  In  particular,  when  the  team  leader  is  distant  to  the  rest  of  the  teams,  team 
members  were  found  to  provide  more  team  leadership.  Also,  based  on  debriefing  materials  and 
verbal  communication,  we  demonstrated  that  leaders  and  teams  adapt  their  structure  both  across 
sessions  and  within  sessions.  We  believe  that  measuring  team  structure  is  an  effective  way  to 
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capture  distant  leadership  and  that  the  findings  associated  with  team  structure  will  have  direct 
implications  for  training  and  team-technology  design. 

Potential  Applications 

We  anticipate  our  efforts  will  fill  a  void  in  the  literature.  In  particular,  our  research  will 
help  us  to  understand  leadership  processes  in  the  context  of  team  performance  and  provide  a 
basis  for  future  training  programs  and  other  interventions  to  enhance  team  performance, 
especially  when  leaders  and  members  are  separated  by  distance. 

First,  team  structures  in  dynamic  task  settings  are  fluid.  To  match  the  changes  in  team 
structure,  distant  leaders  should  be  provided  with  maximum  control  of  communication 
topologies.  Due  to  rapid  changes  found  in  some  work  settings,  teams  may  encounter  dynamic 
task  conditions  in  which  they  have  the  dual  goal  of  successful  training  and  performance.  Our 
experimental  results  suggest  that  team  leaders  adapted  how  they  communicate  with  their  team 
members.  This  ability  to  control  and  rapidly  adapt  configurations  of  communication  channels 
(who  talks  to  whom  and  who  can  hear  what)  is  critical.  The  leaders  in  our  research  showed 
preference  for  a  hierarchical  communication  structure,  but  would  alter  that  structure  when  task 
urgency  was  high  or  the  team  was  unexperienced  working  together.  Secondly,  leadership 
development,  as  built-in  in  the  teams  studied  here,  should  be  viewed  as  a  necessary  adaptation 
strategy  for  those  teams  that  face  highly  fluctuating  task  demands.  In  the  case  of  trauma 
resuscitation  teams,  they  may  have  to  be  split  up  to  treat  more  than  one  patient,  and  necessarily 
junior  members  have  to  enact  leadership.  In  distant  leader, -hip  conditions,  some  of  the  functions 
need  to  be  fulfilled  by  people  other  than  the  leader.  Worded  differently,  when  a  team  is  likely  to 
encounter  highly  dynamic  tasks,  the  team  members  should  expect  to  provide  leadership  even 
though  they  may  not  be  designated  as  the  team’s  leader.  Thirdly,  leadership  should  be  assessed 
not  only  for  the  team  performance  achievement,  but  also  for  allowing  leadership  development 
among  team  members. 


Accomplishments 

We  have  accomplished  all  major  tasks  for  the  project.  The  final  report  was  submitted  to 
ARI.  Through  several  iterations,  the  final  report  reflected  the  main  efforts  of  the  project.  The 
major  empirical  studies  carried  out  in  the  project  were: 

Study  1 :  Team  leadership  in  trauma  -  an  observational  and  interview  study.  This  study 
was  conducted  to  answer  the  broad  questions  of  how  leadership  was  defined  in  the 
context  of  trauma  teams.  We  spent  over  225  hours  at  observing  the  treatment  of 
approximately  175  different  patients.  We  interviewed  10  providers  from  different 
positions  (attending  surgeon,  attending  anesthesiologist,  surgical  resident,  emergency 
medical  resident,  medical  student,  nurse,  and  trauma  technician). 

Study  2:  A  review  of  archival,  videotaped  team  performance.  We  analyzed  120  video 
taped  team  performance  to  extract  leadership  scenarios  encountered  by  trauma  teams. 

Study  3:  A  survey  study  of  leadership  behaviors.  We  developed  a  59-item  survey  of 
leader  behavior  and  collected  the  surveys  from  35  members  of  trauma  care  providers. 
The  survey  was  to  identify  key  leader  behaviors. 
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Study  4:  A  field  experiment  manipulating  leader  location.  This  research  was  the  key  to 
the  project.  We  instituted  a  research  paradigm  in  a  real  patient  care  setting:  Level-I 
trauma  care  to  understand  distant  leadership.  A  battery  of  measurement  tools  was 
deployed  in  127  patient  care  cases  with  89  study  participants  (care  providers  involved  in 
the  study).  Of  the  127  cases,  59  were  videotaped.  Among  the  videotaped  cases,  22  were 
in  distant  leadership  conditions.  We  performed  a  range  of  analyses  from  time-accuracy 
measurement  of  performance,  to  intra-team  communication  measures,  to  case  analysis  of 
152  team  performance  segments. 

Study  5.  In-depth  interview  study:  we  interviewed  16  surgeons  to  understand  leadership 
identification  from  the  point  of  view  of  attending  surgeons  and  residents.  We  gained 
insights  into  important  topics  of  team  effectiveness,  leader  identity,  and  leader  functions. 
Based  on  the  interviews,  we  developed  the  concept  of  team  leadership  systems  to  take 
into  account  the  dynamic  flow  of  leadership. 

Throughout  the  project,  we  published  1 1  peer  reviewed  proceeding  articles  and  book 
chapters  and  made  presentations  in  conferences  of  professional  associations  such  as  Academy 
of  Management,  Society  of  Industrial  and  Organizational  Psychology,  and  Human  Factors  and 
Ergonomics  Society. 

The  theoretic  and  conceptual  efforts  (specific  aims  a-c)  will  be  based  on  existing 
literature  on  team  performance  and  on  the  analysis  of  an  existing  video  library  of  team 
performance  collected  in  a  stressful  environment.  The  cases  from  the  video  library  will  be 
reviewed  to  extract  segments  representative  of  effective  and  ineffective  occurrences  of 
leadership.  These  segments  will  be  used  as  the  basis  for  the  leadership  matrix,  model,  and 
measures.  Prospective  empirical  efforts  (specific  aim  d)  will  utilize  an  already  established  study 
environment  in  a  Level-I  trauma  center.  Comprehensive  audio-video  recordings  will  be  the 
primary  data  collection  method.  Two  modes  of  communication  will  be  randomly  assigned  in 
consecutive  cases.  Leadership  effectiveness  will  be  measured  by  both  team  performance  (speed 
and  accuracy)  and  process  parameters  (decision  processes  and  leadership  control  processes). 
These  dependent  measures  will  be  analyzed  in  relationship  with  several  independent  variables 
including  risk,  urgency,  uncertainty,  task  structure,  and  workload. 

Final  Summary 

Our  research  depicted  a  dynamic,  fluid  team  leadership  in  achieving  two  commonly 
found  goals  of  teams:  performing  and  training.  Teams  adapt  their  leadership  depending  on  task 
conditions,  team  experience,  and  leader  location  (in  the  case  of  distant  leadership).  We  proposed 
the  concepts  of  team  structures  and  team  leadership  systems  to  provide  both  practical  guidance  to 
team  leadership  and  a  theoretical  basis  for  future  team  leadership  research. 
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Leadership,  Team  Processes,  and  Team  Adaptation: 

The  Development  and  Influence  of  Functional  Leadership  Capabilities  on 

Team  Adaptability  to  Adversity 

Contract#:  DASW01-98-K-0005  Contract  Dates:  8/1/98-7/31/03 

Institution:  George  Mason  University  PI:  Stephen  J.  Zaccaro 

Co-PI:  Richard  Klimoski 

Research  Summary 

The  purposes  of  this  research  program  are  (a)  to  examine  in  detail  the  influences  leaders 
have  on  team  processes  contributing  to  team  adaptation,  (b)  to  examine  training  and  development 
principles  that  contribute  to  the  development  of  adaptive  military  leaders  and  teams;  and  (c)  to 
develop  and  examine  the  psychometric  properties  of  a  leader  assessment  tool  that  measures 
personal  qualities  contributing  to  adaptation.  Our  laboratory  research  examines  (a)  leader 
attributes  that  promote  leader  and  team  adaptation,  and  (b)  training  feedback  characteristics  that 
foster  team  adaptation.  Our  field  studies  are  designed  to  (a)  validate  an  assessment  battery 
constructed  to  measure  multiple  leader  attibutes  related  to  leader  flexibility,  (b)  examine  work 
experiences  related  to  the  development  and  emergence  of  attributes  promoting  leader  flexibility, 
and  (c)  test  training  principles  regarding  the  development  of  leader  flexibility  and  (d)  develop  a 
leader  adaptability  training  program. 

Contributions  to  Basic  Research 

The  current  work  advances  basic  research  in  the  areas  of  leader  and  team  adaptability  and 
leader  development.  One  product  of  this  effort  is  a  comprehensive  conceptual  framework  that 
describes  a  career-long  process  of  leader  development.  This  framework  builds  on  previous 
research  in  the  literature  and  incorporates  multiple  sources  of  developmental  experiences  (i.e., 
formal  training,  self-development,  work  assignments,  mentoring,  coaching,  feedback).  Also,  our 
research  provides  support  for  training  principles  designed  to  grow  adaptability  skills  in  both 
leaders  and  team  members.  Our  work  also  identifies  key  leader  and  team  attributes  that  promote 
adaptability  and  training  interventions  that  can  be  used  to  develop  these  attributes.  Based  on 
these  training  principles  and  interventions,  we  have  developed  a  comprehensive  leader 
adaptability  training  program  that  was  administered  at  the  Army  War  College.  Finally,  we  have 
a  growing  database  of  research  that  provides  support  for  proposed  relationships  between  certain 
key  leader  attributes  and  adaptability. 

Potential  Army/Military  Applications 

The  applications  of  this  research  for  the  Army  are  expected  to  be  in  the  areas  of  training 
assessment  and  curriculum.  The  assessment  battery  measuring  leader  flexibility  can  serve  as  a 
very  effective  training  diagnostic  tool  in  multiple  Army  training  settings.  The  assessment  battery 
will  identify  leader  strengths  and  weaknesses  on  various  leader  adaptability  skills  and  attributes 
and  provide  trainers  with  insight  into  which  areas  that  should  be  targeted  in  training.  Further,  the 
field  studies  yield  specific  guidelines  and  curriculum  tools  that  target  the  development  of  leader 
flexibility.  These  guidelines  and  tools  are  expected  to  be  constructed  in  accordance  with  the 
different  developmental  needs  that  emerge  at  various  points  in  an  officer’s  career.  Also,  the 
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adaptability-training  program  that  was  administered  at  the  Army  War  College  should  have  direct 
applicability  to  the  development  of  adaptive  senior  leaders. 

Accomplishments 

We  have  completed  five  experimental  studies.  Our  first  study  examined  the  effects  of 
process  versus  outcome  feedback,  and  individual  versus  team-level  feedback  on  team  adaptation. 
We  also  examined  the  role  of  certain  leader  adaptability  attributes  on  team  processes  and 
performance.  Our  findings  indicated  that  team-level,  process  oriented  feedback,  such  as 
feedback  about  the  team’s  quality  of  communication,  coordination,  strategy,  rather  than  feedback 
regarding  outcomes,  produced  the  strongest  adaptive  responses  in  teams.  However,  leader  and 
team  metacognitive  skills  moderated  the  influences  of  feedback  on  team  adaptability.  In  our 
second  study,  we  investigated  other  dimensions  of  training  feedback,  including  public  versus 
private  feedback,  on  team  efficacy  and  team  mental  models,  as  well  as  team  adaptation.  Our 
findings  indicated  that  shared  mental  models  mediated  the  effects  of  feedback  parameters  on 
team  adaptability.  In  our  third  study,  we  examined  the  role  of  task  variability  and  feedback  in 
training  performance  scenarios  on  adaptability.  Our  findings  were  mixed,  but  overall  they 
indicated  that  process-oriented  feedback  during  training  increased  adaptability  in  team 
performance,  but  varying  a  team’s  task  during  training  did  not  enhance  adaptability  in  team 
performance.  The  findings  for  variability  were  contrary  to  expectations  and  further  work  is 
being  done  to  explore  possible  explanations  (e.g.,  increasing  the  amount  of  task  variability).  Our 
fifth  study  examined  the  effects  of  varaibility  in  greater  detail  and  found  that  the  combination  of 
variability  and  feedback  produced  the  strongest  effects  on  team  adaptability.  In  our  fifth  study, 
we  explored  in  more  detail  the  nature  of  team  mental  models,  finding  that  some  mental  models 
were  idiosyncratic  to  the  team,  while  others  derived  from  common  experiences  of  group 
members. 

For  the  field  research,  we  have  developed  conceptual  models  that  describe  the  influences 
of  work  experiences,  formal  instruction,  mentoring  and  coaching  processes,  feedback  systems, 
and  self-development  experiences  on  individual  attributes  related  to  leader  flexibility.  We  have 
also  developed  an  assessment  battery  that  measures  cognitive,  behavioral,  and  dispositional 
attributes  linked  to  leader  flexibility.  We  have  completed  several  validation  studies,  affirming 
the  psychometric  strengths  of  this  battery.  We  also  examined  the  influence  of  these  attributes  on 
the  relationship  between  reported  developmental  work  experiences  and  gains  in  tacit  knowledge 
in  military  officers.  We  have  administered  this  battery  to  several  classes  at  the  Army  War 
College  (AWC)  and  to  a  sample  of  managers  and  executives  in  several  private  corporations.  We 
found  the  influences  of  developmental  work  experiences  on  leader  tacit  knowledge  depended 
upon  the  levels  of  metacognitive  skill,  cognitive  complexity,  and  tolerance  for  ambiguity 
displayed  by  the  leader.  Finally,  we  have  administed  the  battery  at  the  the  Center  for  Creative 
Leadership  and  the  Air  Coomand  and  Staff  College,  and  collected  ratings  of  adaptability  in 
leadership  simulations.  We  found  that  certain  combinations  of  cognitive,  social,  and 
dispositional  attribues  were  the  strongest  predictors  of  leader  adaptability,  providing  support  for 
the  idea  that  one  should  look  at  patterns  of  attributes  to  predict  leader  adaptability.  We  found 
similar  results  in  data  collected  at  the  AWC. 

In  related  funded  efforts,  we  have  developed  and  pilot-tested  a  proposed  training 
curriculum  at  the  AWC.  This  program  applies  principles  of  adaptability  training  that  were 
derived  from  the  conceptual  models  developed  in  year  one  of  this  effort.  We  have  implemented 
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full  versions  of  this  course  at  AWC  in  Spring,  2001  and  at  ICAF  in  Fall,  2002.  These 
applications  of  our  research  did  not  allow  for  a  training  evaluation.  We  are  currently  exploring 
other  venues  for  such  evaluations.  We  also  revised  components  of  this  training  for  use  in  a 
adaptive  leader  training  program  being  developed  by  Personnel  Decisions  Research  Institute 
(PDRI). 


Final  Summary 

Taken  together,  our  findings  provide  insight  into  training  strategies,  particularly  the 
delivery  and  content  of  training  feedback  and  the  structuring  of  training  scenarios  in  performance 
exercises,  that  foster  leader  and  team  adaptability.  We  have  also  identified  team  and  leader 
attributes  that  can  influence  how  training  design  parameters  influence  training  outcomes.  We 
have  identified  and  validated  sets  of  cognitive,  social,  and  dispositional  attributes  that  predict 
leader  effectiveness  in  dynamic  and  complex  operating  environments.  We  have  also 
demonstrated  that  some  of  these  attributes  moderate  how  developmental  experiences  influence 
some  learning  outcomes.  These  findings  add  to  the  literature  on  adaptive  leadership,  leader 
training  and  development,  and  the  prediction  of  leader  effectiveness,  particularly  in  military 
settings.  This  research  has  provided  prototypes  for  a  number  of  assessment  tools  that  have  begun 
to  be  adapted  for  use  in  particular  military  contexts. 

Link  to  Applied  Research 

This  research  links  to  Dr.  Halpin’s  two  applied  programs,  “Leadership:  Developing 
Leaders  in  a  Changing  Army”  (FY03)  and  “Tomorrow's  LEADERS:  Leading  the  Objective 
Force”  (FY04)  programs. 

The  goals  of  the  first  applied  program,  “Developing  Leaders  in  a  Changing  Army,”  and 
the  goals  of  the  basic  research  align.  Both  focus  on  leader  adaptability  and  both  involve 
developing  leader  assessment  and  self-development  systems  incorporating  cognitive  and 
interpersonal  leadership  skills.  Hence,  work  from  the  basic  program  could  be  directly  applicable 
to  the  work  being  completed  in  the  applied  program.  For  example,  one  goal  of  the  applied 
research  program  in  leader  development  is  to  devise  an  empirically  valid  method  that  embeds 
experience  in  a  self-development  system  for  fast-tracking  Army  leadership  tacit  knowledge  in 
Lieutenants  and  Captains.  Our  work  offers  training  principles  and  tools  for  embedding  leader 
adaptability  development  that  could  be  incorporated  in  the  self-development  system. 

The  second  applied  program,  “Tomorrow's  LEADERS:  Leading  the  Objective  Force,”  is 
linked  to  the  basic  research  efforts  in  developing  adaptive/flexible  leaders.  The  basic  research 
involves  testing  training  principles  for  leader  adaptability  that  include  leadership  assessment  and 
self-development,  and  incorporates  cognitive  and  interpersonal  skills  as  well  as  individual 
dispositions.  Hence,  this  links  directly  to  the  applied  program's  goal  of  developing 
recommendations  for  an  integrated  leadership  assessment  and  self-development  system  for 
officers,  incorporating  both  cognitive  and  interpersonal  skills.  The  work  in  the  basic  program 
could  be  employed  in  the  applied  program’s  assessment  and  self-development  system. 
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BRO  RESEARCH  OBJECTIVE  #3: 


Provide  fundamental  knowledge  for  identifying  and  measuring  the 
attributes  and  skills  that  are  critical  to  soldier  recruiting, 
performance,  and  retention  in  the  transforming  Army. 


Research  in  this  section  is  directed  toward  identifying  and  measuring 
the  aptitudes  and  skills  that  are  unique  to  the  human  performance 
requirements  of  the  Future  Force.  Exploring  the  sociological  and 
psychological  factors  that  could  influence  recruitment,  retention,  and 
Army  performance  are  important  aspects  of  this  research  objective. 
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The  Interactive  Effect  of  Feedback-Sign  and  Regulatory  Focus  on 
Performance  of  Creative  Task  Versus  Error-Detection  Task 

Contract  #:  DASW01-02-K-70512  Contract  Dates:  1 1/4/02-1 1/3/03 

Institution:  The  Hebrew  University  of  Jerusalem  PI:  Avraham  N.  Kluger 

Co-PI:  Dina  Van-Dijk  &  Miriam  Erez 

Research  Summary 

Consider  the  following  question:  Does  positive  (negative)  feedback  encourage  or 
discourage  performance?  This  seemingly  simple  question  defied  simple  answers.  A 
comprehensive  meta-analysis  showed  that  positive  feedback  had  a  positive  effect  on 
performance  in  some  studies,  while  in  other  cases  it  had  no  effect,  or  a  negative  effect,  and  vice 
versa  for  negative  feedback  (Kluger  &  DeNisi,  1996).  To  account  for  these  variable  feedback- 
sign  (positive  or  negative)  effects,  Van-Dijk  and  Kluger  (in  press)  suggested  applying  self¬ 
regulation  theory  (Higgins,  1997,  1998).  This  theory  distinguishes  between  promotion  and 
prevention  focus  of  motivation.  Van-Dijk  and  Kluger  suggested  that  relatively  high  levels  of 
motivation  are  induced  either  by  failure  under  prevention  focus  (failure  to  meet  obligations)  or 
by  success  under  promotion  focus  (fulfilling  a  desire).  Thus,  the  theoretical  answer  to  the 
question  -  does  feedback  encourage  performance  -  is  it  depends.  It  depends  on  whether  the 
person  is  eager  to  perform  the  task  or  is  obliged  to  do  so. 

In  initial  experiments  testing  this  explanation,  Van-Dijk  and  Kluger  found  that  people 
with  a  prevention  focus  are  more  highly  motivated  by  negative  feedback  while  those  with  a 
promotion  focus  are  motivated  by  positive  feedback.  However,  the  Van-Dijk  and  Kluger  studies 
(in  press)  were  preliminary  in  nature.  They  used  hypothetical  scenarios  to  gauge  motivation,  but 
had  no  performance  measures.  In  addition,  they  did  not  consider  the  task  characteristics  which 
could  be  a  decisive  factor  in  inducing  either  promotion  or  prevention  focus.  Thus,  the  goal  of  this 
project  was  to  test  whether  the  interaction  of  feedback  sign  with  regulatory  focus  affects  actual 
performance,  and  whether  the  nature  of  the  task  influences  the  situational  regulatory  focus  - 
promotion  versus  prevention.  Specifically,  our  research  examines  the  effect  of  feedback  sign  on 
the  performance  of  a  task  considered  to  induce  situational  promotion  focus  (creativity  task)  and 
on  the  performance  of  a  task  considered  to  induce  situational  prevention  focus  (error-detection 
task).  Briefly,  it  was  expected  that  negative  feedback  will  improve  the  performance  on  error- 
detection  tasks,  and  positive  feedback  will  improve  the  performance  on  creative  tasks.  Yet,  this 
interaction  was  expected  to  further  depend  on  individual  differences  in  self-regulation  foci:  the 
benefit  of  positive  feedback  for  the  performance  of  creative  tasks  was  expected  only  among 
participants  with  a  chronic-promotion  focus. 

Contributions  to  Basic  Research 

This  project  provides  an  explanation  for  the  complex  effect  that  feedback  sign  is  likely  to 
have  on  motivation  and  hence  on  performance.  It  points  out  an  interesting  phenomena  related  to 
negative  feedback.  This  project’s  hypothesis  suggests  that  under  prevention  focus  negative,  but 
not  positive,  feedback  would  best  enhance  performance.  Yet,  vast  literature  has  documented  the 
debilitating  effect  of  negative  feedback  on  self-efficacy  and  as  a  result  on  performance. 
Specifically,  existing  theories  concentrate  on  the  mecahnism  of  expectation  for  future 
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performance  as  a  key  for  understanding  feedback-sign  effect.  Our  work  shows  that  expectation 
is  only  part  of  the  story  where  value  or  desirability  is  another  part.  That  is,  failure  can  not  only 
reduce  expectations  for  success,  it  can  also  increase  the  desirability  of  future  success.  Thus,  our 
work  may  pave  the  way  to  identify  means  to  reap  the  motivational  benefits  of  negative  feedback 
while  avoiding  the  traps  of  debilitation  associated  with  failure. 

Potential  Army/Military  Applications 

Our  hypotheses  suggest  that  no  feedback  system  can  fit  all.  That  is,  certain  military 
occupations  that  largely  require  high  vigilance  may  be  best  staffed  with  soldiers  who  have  a 
strong  chronic  prevention  focus  (which  may  be  measured  by  values,  vocational  interests,  and 
perhaps  with  personality  traits  such  as  conscientiousness).  Such  occupations  may  include 
monitors  of  radar  screens,  guards,  maintenance  personnel,  etc.  These  soldiers’  performance  may 
benefit  most  from  training  that  emphasizes  punishment  for  poor  performance,  and  from  a 
performance  appraisal  system  that  emphasizes  error-free  performance,  which  prevents  the 
promotions  of  soldiers  with  records  of  frequent  errors  and  disciplinary  records.  In  contrast,  other 
military  occupations  that  requires  a  state  of  eagerness,  for  example  among  elite  fighting  units, 
may  be  best  staffed  with  soldiers  that  have  a  strong  chronic  promotion  focus,  and  their  training 
and  evaluations  may  benefit  most  from  awards,  prizes  and  recognition  of  excellence. 

The  application  of  our  recommendation  would  be  relatively  easy  to  implement.  The 
manipulation  of  veridical  feedback  so  it  is  perceived  either  as  negative  or  positive  can  be 
achieved  with  framing:  messages  can  be  easily  framed  by  emphasizing  either  the  half-full  glass 
or  the  half-empty  glass  (Levin,  Schneider,  &  Gaeth,  1998).  This  was  already  found,  but  not 
explained  theoretically,  by  Wade  (1974).  He  instructed  people  to  monitor  either  errors  versus  or 
successes  in  a  vigilance  task  and  found  that  monitoring  errors  produced  better  performance. 
Using  this  ease  of  manipulation,  we  recommend  emphasizing  the  negative  aspects  of 
performance  when  the  behavior  in  question  is  about  prevention.  For  example,  a  tank 
maintenance  task  should  be  monitored  by  the  number  of  problems  found  in  the  inspection  of  the 
tank,  and  not  by  the  number  of  problem-free  tanks  successfully  maintained.  In  contrast,  when 
monitoring  training  of  officer  cadets  being  instructed  in  suggesting  a  battle  plans,  performance 
should  monitor  the  number  of  excellent  ideas  generated  by  the  trainees  rather  than  monitoring 
the  number  of  poor  ideas.  Of  course,  in  many  military  occupations  one  needs  both  prevention 
and  promotion  abilities.  Under  these  circumstances  a  complex  schedule  of  feedback  sign  will  be 
needed.  Yet,  commanders  can  be  trained  to  decode  the  situations  that  are  more  likley  to  benefit 
from  praise  versus  criticism.  The  practical  rule  that  emerges  from  our  theory  is  “punish  failure 
to  prevent  errors,  but  do  not  praise  error  detections;  praise  success  in  creative  tasks,  but  do  not 
punish  poor  performance  on  these  tasks”. 

Accomplishments 

Our  experiment  corroborated  our  hypotheses.  Prior  to  the  experiment,  all  participants 
filled  out  a  value  questionnaire  that  was  used  to  assess  their  chronic  self-regulatory  focus.  In  the 
experiment,  half  of  the  participants  worked  on  an  error-detection  task  (finding  errors  in 
arithmetic  calculations)  and  the  other  half  worked  on  a  creativity  task  (designing  an 
advertisement  using  PowerPoint  software).  After  working  on  the  task,  participants  were 
provided  with  a  bogus  feedback  indicating  that  so  far  they  were  performing  either  well  or  poorly. 
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After  receiving  feedback,  participants  were  asked  to  continue  working  on  their  task.  Our  key 
interest  was  in  the  effect  of  feedback  sign  on  post-feedback  performance  change. 

As  expected,  the  effect  of  feedback  sign  depended  both  on  the  nature  of  the  task  and  on 
the  values  (chronic  self-regulatory  focus)  of  the  participants.  Specifically,  (a)  in  the  error- 
detection  task  performance  was  higher  after  negative  feedback  than  after  positive  feedback, 
whereas,  in  the  creative  task,  performance  was  higher  after  positive  feedback  than  after  negative 
feedback;  (b)  for  participants  low  in  chronic-promotion  focus,  performance  was  higher  after 
negative  feedback  than  after  positive  feedback,  whereas,  for  people  high  in  chronic-promotion 
focus  performance  was  higher  after  positive  feedback  than  after  negative  feedback;  and  (c) 
people  low  in  chronic-promotion  focus  were  more  likely  to  show  improved  performance  after 
failure  than  after  success,  regardless  of  the  task.  In  contrast,  the  reactions  of  participants  high  in 
chronic-promotion  focus  to  feedback-sign  depended  on  the  task:  they  improved  performance 
after  failure  on  the  error-detection  task  and  after  success  on  the  creativity  task. 

Final  Summary 

The  results  of  our  experiment  further  corroborated  the  theory  advanced  by  Van-Dijk  and 
Kluger.  These  results  enable  us  to  predict  that  positive  feedback  would  be  more  effective  for 
people  with  a  promotion  focus,  whereas  negative  feedback  would  be  more  effective  for  people 
with  a  prevention  focus.  Furthermore,  our  experiment  suggests  that  promotion  focus  versus 
prevention  focus  could  be  determined  by  individual  differences  (we  measured  by  values)  and  by 
the  nature  of  the  task  (we  manipulated  error-detection  tasks  versus  creativity  task).  Our  findings 
suggest  that  no  feedback  system  can  fit  all.  That  is,  organizations,  including  the  military,  should 
maintain  congruency  between  feedback  system,  task  type  and  employees  characteristics. 
Specifically,  organizations  (or  certain  units  within  organizations)  that  largely  require  high 
vigilance  such  as  certain  military  jobs  (e.g.,  radar  operator),  security  companies,  and  banks,  may 
be  best  staffed  with  employees  that  have  strong  chronic  prevention  focus,  because  these  jobs 
usually  emphasize  the  negative  outcome  of  errors  and  mistakes.  Performance  of  prevention- 
focused  employees  may  benefit  most  from  training  that  emphasize  avoiding  poor  performance, 
and  by  performance  appraisal  systems  that  emphasize  error-free  performance,  which  can  prevent 
the  promotions  of  employees  with  records  of  frequent  errors  and  disciplinary  records.  In 
contrast,  other  organizations  that  require  a  state  of  eagerness,  for  example,  intelligence  units,  and 
commando  units,  may  be  best  staffed  with  employees  (officers)  that  have  a  strong  chronic 
promotion  focus.  Performance  of  promotion-focused  employees  may  benefit  most  from  awards, 
prizes  and  recognition  of  excellence.  Finally,  our  project  calls  for  understanding  the  interplay 
between  feedback  sign  and  promotion  and  prevention  in  complex  tasks  that  may  require  the 
ability  to  shift  from  promotion  to  prevention.  For  example,  the  performance  of  an  excellent 
soldier  in  a  commando  unit  may  require  both  the  ability  to  prevent  trouble  and  the  creativity 
needed  to  improvise  performance  in  the  field.  Thus,  the  conclusions  that  can  be  drawn  from  our 
project  is  that  feedback  (whether  positive  or  negative)  should  be  tailored  for  the  person/task 
combination  and  that  the  combination  for  person/task  in  terms  for  promotion  and  prevention 
requires  further  theoretical  and  empirical  work  -  a  path  that  was  opened  by  the  current  project. 
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Contract#:  DASW01-00-K-0018 
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Research  Summary 

One  of  the  most  daunting  challenges  for  work  teams  is  personnel  turnover  (i.e.,  exit  of 
current  members  and/or  entry  of  new  members).  Turnover  can  have  profound  consequences  for 
team  performance,  because  it  alters  both  the  knowledge  distribution  in  the  team  and  the  relations 
among  team  members.  When  current  members  leave,  those  who  remain  must  renegotiate  their 
responsibilities  for  storing  and  sharing  information.  When  new  members  enter,  they  must 
acquire  knowledge  about  their  roles  and  about  others’  competencies  and  responsibilities.  In 
some  cases  (e.g.,  when  the  team  is  performing  poorly  and  newcomers  possess  task  knowledge), 
newcomers  can  introduce  changes  in  team  behavior  and  thereby  enhance  team  adaptability. 

This  research  program  utilizes  two  team  tasks  (decision  making  and  production)  and 
employs  two  methodologies  (laboratory  experimentation  and  computer  simulation).  Three 
related  lines  of  work  are  being  conducted.  Studies  using  the  experimental  version  of  the 
decision-making  task  investigate  the  conditions  under  which  newcomers  stimulate  adaptation 
and  innovation  in  work  teams.  Studies  using  the  experimental  version  of  the  production  task 
investigate  how  newcomers  affect  the  team’s  transactive  memory  system  —  a  shared  mental 
model  about  how  task  competencies  are  distributed  across  team  members.  The  simulation 
studies,  which  employ  powerful  computer  programs  that  have  proven  useful  in  modeling  group 
and  organizational  performance,  extend  the  laboratory  work  in  various  ways.  For  example  the 
simulation  work  extends  the  laboratory  research  by  investigating  the  impact  of  turnover  in  larger 
social  units  and  over  longer  time  periods. 

Contributions  to  Basic  Science 

Only  a  handful  of  studies  have  examined  personnel  turnover  in  teams.  These  studies 
suggest  that  turnover  typically  disrupts  team  performance,  but  is  more  deleterious  under  some 
conditions  than  others.  Because  these  studies  vary  widely  on  conceptual  as  well  as 
methodological  grounds,  their  results  are  difficult  to  integrate  in  a  theoretically  satisfying 
manner.  Therefore,  a  major  goal  of  the  present  research  program  is  to  provide  a  stronger 
theoretical  framework  for  understanding  the  consequences  of  turnover  (positive  as  well  as 
negative)  for  work  teams.  One  product  of  our  work  is  a  comprehensive  theoretical  framework 
for  understanding  the  conditions  under  which  newcomers  can  alter  the  work  practices  of  teams 
they  enter.  In  addition,  our  simulation  work  makes  important  headway  in  integrating  knowledge 
about  turnover  in  groups  and  organizations,  as  well  as  suggesting  a  number  of  interesting 
hypotheses  for  investigation  in  both  laboratory  and  natural  groups.  Finally,  we  have  obtained  a 
substantial  corpus  of  laboratory  and  simulation  data  that  enrich  the  literature  on  how  teams  adapt 
to  the  challenge  of  personnel  turnover. 
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Potential  Army/Military  Applications 


Personnel  turnover,  a  fact  of  life  in  the  Army,  can  have  several  consequences.  For  example, 
when  membership  change  is  planned  and  new  personnel  are  well-trained,  turnover  may  have 
minimal  effects  on  team  performance.  In  contrast,  when  membership  change  is  unplanned 
and/or  new  personnel  are  not  prepared  for  their  mission,  turnover  may  severely  degrade  team 
performance.  Finally,  when  team  performance  is  already  low  (e.g.,  because  well-learned  task 
routines  are  not  suited  to  a  new  operational  environment),  turnover  may  significantly  enhance 
team  performance.  Our  work  suggests  several  strategies  for  reducing  the  costs  and  enhancing 
the  benefits  of  membership  change  in  teams.  For  example,  our  research  on  transactive  memory 
systems  suggests  that  warning  teams  that  turnover  will  occur,  combined  with  information  about  a 
newcomer’s  skills,  partially  mitigates  the  negative  consequences  of  turnover.  Our  work  on 
newcomer  innovation  suggests  that  allowing  teams  to  choose  their  task  strategy  reduces  their 
willingness  to  adopt  new  strategies  in  the  face  of  failure.  And  our  simulation  studies  suggest 
how  turnover  might  be  used  to  destabilize  “enemy”  groups,  which  is  relevant  to  counter¬ 
terrorism  situations. 


Accomplishments 

In  our  laboratory  work  on  newcomer-induced  innovation,  we  used  an  air-surveillance 
task  in  which  three-person  teams  (a  commander  and  two  specialists)  monitored  planes  flying 
through  a  simulated  airspace.  The  first  study,  using  existing  data,  assessed  the  impact  of 
newcomer  ability  and  status  on  team  performance  and  strategy-relevant  communication.  Teams 
that  had  worked  on  the  task  either  experienced  turnover  (involving  a  new  commander  or 
specialist  from  another  team)  or  did  not.  As  predicted,  turnover  did  not  substantially  influence 
team  performance,  presumably  because  the  team  task  was  highly  routinized  and  newcomers  were 
experienced.  Also  as  predicted,  high-status,  high-ability  newcomers  improved  team  performance 
more  than  did  other  newcomers,  and  newcomer  entry  stimulated  strategy-relevant 
communication,  a  potentially  important  mechanism  of  newcomer  influence.  Our  first  study 
directly  investigating  newcomer  innovation  manipulated  whether  the  team  (a)  chose  or  was 
assigned  its  initial  task  strategy  and  (b)  failed  or  succeeded  when  using  this  strategy  prior  to  the 
newcomer’s  entry.  After  performance  feedback,  a  confederate  replaced  one  subordinate.  The 
confederate  suggested  a  new  strategy  for  the  next  work  shift.  As  predicted,  the  newcomer 
produced  more  innovation  when  teams  did  not  choose  their  strategy  and  subsequently  performed 
poorly.  Our  second  innovation  study,  still  in  progress,  is  testing  the  hypothesis  that  subordinate 
newcomers  who  behave  assertively  will  produce  more  innovation  than  nonassertive  newcomers 
following  team  failure,  but  not  team  success. 

We  also  conducted  two  laboratory  experiments  investigating  how  turnover  affects  teams’ 
transactive  memory  systems,  using  a  task  in  which  teams  assembled  transistor  radios.  In  the  first 
study,  some  teams  were  warned,  prior  to  group  training,  that  turnover  would  occur  at  the  start  of 
team  testing,  whereas  other  teams  were  not  warned.  In  addition,  some  teams  in  each  warning 
condition  later  experienced  turnover  (replacement  of  a  current  member  by  a  newcomer  who  had 
been  trained  alone),  whereas  others  did  not.  We  expected  stronger  transactive  memory  and 
better  team  performance  when  turnover  did  not  occur  than  when  it  did,  whether  or  not  a  warning 
was  given.  When  turnover  occurred,  we  expected  the  presence  vs.  absence  of  warning  would 
differentially  affect  transactive  memory  and  performance.  Results  did  not  support  our 
hypotheses,  apparently  because  newcomers  tried  hard  to  learn  the  task,  perhaps  to  impress 


59 


oldtimers,  and  oldtimers  worked  hard  after  newcomer  entry,  perhaps  to  impress  newcomers.  In 
the  second  study,  after  group  training,  teams  experienced  one  of  the  following:  no  turnover, 
turnover  without  warning,  or  turnover  with  warning.  In  the  latter  case,  only  oldtimers  (people 
who  remained  after  turnover),  only  the  newcomer,  or  both  oldtimers  and  the  newcomer  received 
information  about  the  newcomer’s  skills.  Preliminary  results  suggest  that,  as  expected,  no 
turnover  teams  performed  better  than  turnover  teams  with  warning  (and  information  about  the 
newcomer),  which  in  turn  performed  better  than  turnover  teams  without  warning. 

In  addition,  we  developed  computational  models  for  simulating  group  performance  in 
both  the  newcomer  innovation  and  transactive  memory  (TM)  domains  and  ran  virtual 
experiments  to  validate  and  extend  the  models  against  laboratory  and  field  data.  We  found,  for 
example,  that  TM  is  most  valuable  in  groups  of  16-30  members  and  in  volatile  task 
environments.  In  addition,  we  examined  how  IT  affects  the  development  of  TM  and  mitigates 
the  effects  of  turnover,  finding,  for  example,  that  referential  databases  can  substitute  for  human 
TM  and  that  IT  can  facilitate  handover  when  newcomers  enter  a  group.  We  also  developed 
metrics  for  identifying  critical  individuals  in  networks,  focusing  on  agents’  relations  to  others 
and  knowledge  (degree  of  centrality,  access  to  information,  ability  to  direct  others)  and  their 
privileged  position  (exclusivity  in  task  or  knowledge).  Finally,  we  ran  virtual  experiments  on 
network  healing,  examining  the  implications  of  removing  various  kinds  of  agents  (e.g.,  people  in 
specialized  positions,  newcomers).  Results  indicate  that  the  underlying  network  structure 
controls  the  network’s  ability  to  recover,  the  speed  of  recovery,  and  relative  impact  of  removing 
different  types  of  agents. 

Finally,  we  wrote  several  theoretical  and  review  papers  on  topics  related  to  personnel 
turnover,  including  newcomer  innovation  in  work  teams,  group  reactions  to  loyalty  and 
disloyalty,  scientific  collaboration,  role  transitions,  and  group  composition. 

Final  Summary 

Our  research  has  identified  several  factors  that  influence  the  impact  of  personnel  turnover 
on  team  performance.  In  addition,  it  has  provided  information  about  the  conditions  under  which 
newcomers  can  influence  existing  work  practices  in  teams.  These  findings  add  to  the  literatures 
on  team  performance,  newcomer  innovation,  and  transactive  memory.  They  also  demonstrate 
the  synergies  that  can  result  when  investigators  from  different  disciplines  (social  psychology  and 
sociology)  use  different  methodologies  (laboratory  research  and  computer  simulation)  to 
examine  common  problems. 
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Enlistment  Properties  of  University  Students 

Contract  #:  1435-01-01-CT-31151  Contract  Dates:  6/1/01-5/31/04 

Institution:  Northwestern  University  PI:  Charles  Moskos 

Research  Summary 

Heretofore  Army  enlistment  incentives  have  largely  focused  on  the  recent  high  school 
graduate.  The  college  graduate  has  never  been  viewed  as  major  source  of  enlistees.  This 
research  was  the  first  ever  to  survey  undergraduates  on  campus  on  enlistment  propensities. 
Surveys  were  conducted  at  a  cross-section  of  universities:  Northwestern  University  (selective 
private  university),  University  of  Arizona  (major  state  university),  University  of  Califomia-Los 
Angeles  (major  state  university),  and  University  of  Illinois-Chicago  (predominantly  working- 
class  students).  Over  1,700  undergraduates  completed  surveys  at  these  four  universities.  The 
surveys  were  conducted  during  the  academic  years  2000-01  and  2002-03. 

Surprisingly,  the  differences  in  enlistment  propensity  between  the  universities  were 
remarkably  similar  despite  the  variation  in  the  socio-economic  background  of  the  four 
universities.  The  core  finding  was  that  there  is  a  definite,  albeit  limited,  undergraduate 
population  that  would  be  amenable  to  a  short  enlistment  option  —  18  months  —  coupled  with 
educational  benefits.  Enlistment  propensity  was  positively  correlated  with  having  a 
friend/relative  with  military  experience,  student  debt,  and  the  probability  of  serving  overseas. 

It  is  noteworthy  to  mention  that  enlistment  propensity  was  not  correlated  with  either 
political  orientation  or  knowledge  of  military  life.  The  only  major  sociological  difference  was 
that  men  were  about  twice  more  likely  to  show  an  enlistment  propensity  than  women,  although 
the  women  were  three  times  more  likely  than  the  men  to  show  an  enlistment  propensity  for 
civilian  national  service. 


Contributions  to  Basic  Science 

The  findings  support  a  dual-market  approach  in  recruitment.  This  contrasts  with  the 
prevailing  labor-market  paradigm  that  dominates  the  research  of  Department  of  Defense  contract 
consultants.  Recruitment  policies  must  incorporate  social  psychological  theories.  In  particular, 
the  surveys  show  that  length  of  enlistment  term  is  more  of  a  factor  than  monetary  incentives 
among  college  undergraduates. 

The  data  also  indicate  that  models  of  recruitment  that  rely  only  on  econometric  analyses 
are  deficient.  It  is  time  to  introduce  social  psychological  variables  in  recruitment  analyses.  The 
eminent  psychologist  Leon  Festinger  developed  a  "comparison  theory"  in  the  1950’s.  This  is  to 
say  that  humans  evaluate  themselves  not  so  much  by  objective  standards  such  as  monetary 
rewards,  but  by  comparisons  with  people  around  them.  Likewise,  the  noted  social  psychologist 
James  Coleman  rejected  the  economist's  "individualistic  premises"  and  argued  for  the 
importance  of  relations  among  people  in  making  life  choices. 
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Similarly,  the  concept  of  "reference  groups"  would  enlighten  recruitment  studies.  This  is 
to  say  that  people  get  their  cues  from  meaningful  groups  rather  than  a  generalized  other.  For 
recruitment  purposes,  nothing  would  be  more  meaningful  than  one’s  friends  joining  the  Army. 

Potential  Army/Military  Applications 

The  basic  finding  is  that  for  college  graduates,  shorter  enlistments  are  more  favorable. 
Some  35  percent  of  the  surveyed  students  expressed  a  favorable  enlistment  propensity  for  an  18- 
month  enlistment  coupled  with  educational  benefits  that  would  reduce  student  loans. 

A  significant  finding  is  the  high  percentage  of  college  graduates  who  now  plan  go  on  to 
graduate  school.  Forty  percent  of  the  surveyed  students  indicated  such  intentions  right  after 
graduation.  In  point  of  fact,  a  higher  percentage  of  youth  now  go  on  to  graduate  school  than 
went  to  undergraduate  school  in  the  immediate  post-WWII  years.  These  new  realities  argue  for  a 
rethinking  of  GI  Bill-type  programs  to  include  appealing  to  future  graduate  students. 

From  this  standpoint,  a  break  between  a  bachelor’s  degree  and  graduate  studies  might  be 
a  new  incentive  for  military  recruitment.  For  college  youth,  the  citizen-soldier  rather  than  a 
military  career  is  where  recruitment  appeals  must  focus.  For  this  population,  the  emphasis 
should  be  placed  on  military  service  as  a  rewarding  change  of  pace  and  interim  experience 
between  undergraduate  and  graduate  school  or  between  school  and  career. 

Other  applications  of  the  research  for  Army  enlistment  include:  (1)  rely  more  on  veterans 
who  were  not  career  soldiers  for  recruiters,  (2)  aim  advertisements  at  veterans  to  influence 
younger  relative  and  friends  to  enlist,  (3)  consider  having  university  newspaper  editors  visit 
troops  in  the  field  and  do  follow-up  stories. 

Accomplishments 

Starting  this  fall,  the  Army  will  introduce  a  modified  form  of  the  18-month  enlistment 
option.  In  some  degree,  the  enabling  legislation  specifying  this  option  was  influenced  by  the 
research  reported  herein. 


Future  Plans 

Follow-up  surveys  will  be  conducted  at  selected  university  campuses.  As  always,  the 
principal  investigator  will  conduct  research  on  sociological  topics  requested  by  senior  military 
officers  and  Defense  Department  officials. 
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Comprehension  and  Memory  of  Spatial  and  Temporal  Event  Components 

Contract  #:  DASW01 -02-K-0003  Contract  Dates:  05/01/02-04/04/04 

Institution:  University  of  Notre  Dame  PI:  Gabriel  A.  Radvansky 

Research  Summary 

When  people  are  presented  with  information  about  events  in  the  world,  in  order  to 
successfully  comprehend  and  remember  that  information,  they  need  to  be  able  to  deal  with 
changes  that  occur.  That  is,  situations  and  events  are  constantly  and  dynamically  changing  over 
time.  As  such,  people  must  be  able  to  identify  when  the  changes  that  have  occurred,  and  to 
update  their  understanding  of  the  event  in  a  way  that  takes  these  changes  into  account.  There  are 
many  components  of  a  situation  that  can  dynamically  change.  The  focus  of  this  project  is  on 
changes  in  spatial  and  temporal  information  about  an  event.  That  is,  when  there  has  been  a  shift 
in  space  (an  entity  is  in  a  new  location)  or  time  (a  significant  period  of  time  has  passed),  people 
need  to  know  what  information  from  the  previous  situation  will  still  be  relevant  and  what 
information  is  not.  For  example,  if  a  person  learns  that  someone  has  traveled  from  New  York  to 
London,  after  noticing  that  this  change  has  occurred,  in  order  to  properly  understand  the  situation 
the  person  should  mentally  keep  track  of  the  fact  that  the  traveler  had  brought  a  computer  disk 
with  classified  information  with  her,  and  that  she  has  left  behind  her  book  of  contacts.  We  are 
making  this  assessment  of  spatial  and  temporal  updating  using  our  understanding  of  how  people 
create  and  manipulate  their  understanding  and  memory  of  the  world  by  using  mental 
representations  called  mental  models.  By  looking  at  the  success  that  people  are  able  to  track  and 
update  this  information  we  hope  to  be  able  to  better  identify  not  only  those  circumstances  where 
people  are  able  to  do  this  correctly,  but  also,  more  importantly,  those  circumstance  where  people 
are  not  successful  at  updating  this  information,  and  ways  to  correct  that  deficiency. 

Contributions  To  Basic  Science 

This  project  will  provide  us  with  a  more  advanced  understanding  of  how  people  track 
changes  in  descriptions  of  situations  that  occur  in  the  world.  This  research  follows  several  recent 
developments  in  mental  model  theories  and  extends  them  by  looking  more  precisely  and  the 
stages  that  are  involved  in  updating  understanding.  This  more  precise  analysis  will  allow  for  a 
clearer  understanding  of  how  people  comprehend,  monitor  and  remember  changes  in  events  and 
situations  in  the  world  based  on  the  information  they  receive.  From  this  data,  we  hope  to  develop 
a  means  of  identifying  when  people  are  having  trouble  updating  event  information,  and  provide  a 
more  targeted  means  of  correcting  those  problems. 

Potential  Army  /  Military  Applications 

The  potential  applications  of  this  research  for  the  Army  are  in  the  areas  of  training  and 
system  interfaces.  In  the  area  of  training,  this  research  will  provide  insight  into  the  specific 
difficulties  that  people  have  updating  their  understanding  of  events.  Knowing  in  what  ways 
people  have  trouble  tracking  changes  in  space  and  time  will  allow  for  the  development  of 
procedures  that  would  directly  target  such  deficits.  In  addition,  because  the  modem  soldier  needs 
to  interact  with  sophisticated  technologies  that  involve  tracking  various  entities  through  time  and 
space  (whether  they  be  enemy  forces,  one’s  own  supplies  or  friendly  forces),  knowing  the 
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capabilities  and  limits  of  human  event  comprehension  and  memory  will  help  in  the  design  of 
these  systems.  Spatial  and  temporal  information  that  is  readily  tracked  by  people  would  need 
either  minimal  or  no  feedback  from  the  system.  However,  those  aspects  of  spatial  and  temporal 
processing  that  are  more  difficult  would  be  a  higher  priority  for  that  system,  in  terms  of 
augmenting  the  soldier’s  current  capabilities. 

Accomplishments 

We  have  completed  data  collection  looking  at  people’s  ability  to  process  spatial  shifts. 
That  is,  when  people  are  comprehending  information  and  a  shift  in  space  occurs,  as  when  a 
person  moves  from  one  location  to  another,  how  well  are  people  able  to  process  this  information, 
and  how  are  the  different  abilities  related  to  one  another?  We  have  looked  at  a  number  of 
measures  that  assess  spatial  processing.  The  first  is  the  ability  to  detect  that  a  change  has 
occurred.  The  second  is  to  establish  a  representation  of  the  new  location.  The  third  is  to  be  able 
to  keep  activated  information  from  the  previous  location  that  will  be  relevant  in  the  new  location, 
(e.g.,  if  a  person  brought  an  object  to  the  new  location,  knowledge  of  this  object  should  continue 
to  be  available).  The  fourth  and  final  one  is  to  be  able  to  remove  from  the  current  stream  of 
thought  any  knowledge  about  entities  that  were  present  in  the  initial  location,  but  are  not  in  the 
new  location.  These  spatial  processes  were  measured  using  a  number  of  dependent  variables  to 
converge  on  a  coherent  picture  of  the  updating  process. 

The  results  of  this  work  have  demonstrated  that  overall  people  show  the  sorts  of  effects 
that  one  would  expect.  They  effectively  process  information  about  new  locations,  knowledge 
about  entities  that  are  moved  to  the  new  location  continue  to  stay  available,  whereas  knowledge 
about  entities  that  are  part  of  the  prior  location  become  less  available.  Moreover,  the  ability  to 
successfully  update  one’s  understanding  of  the  event  in  one  area  is  related  to  the  ability  to  update 
in  other  ways.  Currently  we  are  further  exploring  in  these  data  how  these  spatial  updating 
processes  are  related  to  more  general  cognitive  measures  that  were  collected  during  the  study, 
including  working  memory  span,  processing  speed,  and  general  reasoning. 

Future  Directions 

Currently  we  are  following  two  lines  of  experimental  inquiry.  The  first  is  that  we  are 
beginning  to  explore  the  ability  of  people  to  update  temporal  information  about  a  situation.  This 
will  parallel  our  efforts  looking  at  the  ability  to  update  spatial  information.  That  is,  we  will  be 
looking  at  people’s  abilities  to  detect  temporal  shifts,  to  create  a  new  temporal  framework  for 
when  such  shifts  occur,  to  maintain  information  that  continues  to  be  relevant  after  a  temporal 
shift  has  occurred,  and  finally,  to  remove  information  from  the  current  stream  of  thought  that  is 
no  longer  relevant  because  of  the  temporal  shift.  The  other  line  will  continue  looking  at  the 
ability  to  process  spatial  shifts.  However,  rather  than  relying  on  strictly  language-based 
materials,  we  will  have  people  traveling  through  desktop  virtual  reality  environments.  The  aim  of 
this  line  of  research  will  be  to  look  at  how  discoveries  in  language-based  understand  of  situations 
and  events  can  be  extended  to  more  realistic  situations.  The  desktop  virtual  reality  environment 
is  helpful  because  it  allows  us  to  construct  various  aspects  of  a  spatial  environment  to  adequate 
test  their  influence  on  thought. 
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Working  Memory  Influences 
on  Long-Term  Memory  and  Comprehension 


Contract  #:  DASW01-99-K-0001  Contract  Dates:  06/15/99-06/14/02 

Institution:  University  of  Notre  Dame  PI:  Gabriel  A.  Radvansky 

Research  Summary 

The  purpose  of  this  research  program  was  to  look  at  the  role  of  working  memory  abilities 
in  the  comprehension  and  memory  of  events  and  situations.  Complex  sets  of  information  about 
events  in  the  world  need  to  be  readily  understood  by  the  people  entrusted  with  that  information. 
This  includes  the  ability  to  understand  descriptions  of  events  occurring  in  some  other  part  of  the 
world,  or  the  need  to  understand  and  accurately  remember  instructions  about  how  to  perform 
(what  to  do)  under  various  circumstances  when  the  need  arises.  To  understand  information  well, 
a  person  must  be  able  to  successfully  comprehend  the  nature  of  the  events  that  are  being  referred 
to  by  the  descriptions  that  have  been  provided.  The  simple  retention  of  a  collection  of  individual 
pieces  of  information  is  insufficient.  A  person  must  be  able  to  properly  relate  that  information  to 
external  circumstances.  The  primary  goal  of  this  project  was  to  assess  how  an  individual’s 
cognitive  abilities  can  be  used  to  predict  future  performance  on  comprehension  and  long-term 
memory  tasks.  Specifically,  this  project  focuses  on  how  working  memory  (that  portion  of 
cognition  where  information  is  actively  manipulated)  influences  higher-level  comprehension  and 
the  long-term  memories  that  result  from  this  comprehension.  These  long-term  memories  are 
often  called  mental  models  because  they  are  mental  simulations  of  a  set  of  circumstances  that 
could  exist  in  the  world.  This  project  assessed  the  degree  to  which  the  successful  processing  of 
mental  models  is  predicted  by  traditional  measures  of  cognitive  ability,  or  whether  this 
processing  can  be  better  predicted  by  measures  aimed  more  directly  at  how  these  representations 
are  created  and  retrieved.  Several  studies  were  conducted  that  looked  at  various  ability  tests  that 
can  be  used  to  predict  future  performance  in  processing  event  and  situation  information.  This  is 
important  because  our  knowledge  about  events  in  the  world  relies  more  on  the  mental  models  we 
create  than  on  lower  level  representations,  such  as  propositional  codes.  Mental  models  are  a 
means  of  capturing  and  coordinating  complex  sets  of  information  about  circumstances  and 
events. 


Contributions  To  Basic  Science 

This  project  provides  us  with  a  more  advanced  understanding  of  working  memory 
capacity  and  processes  as  well  as  the  role  these  play  in  the  various  aspects  of  comprehension  and 
long-term  memory  of  complex  sets  of  information  about  events  in  the  world.  This  research 
follows  and  integrates  long-time  traditions  in  each  of  these  areas  of  cognition.  Furthermore,  a 
better  understanding  will  be  gained  of  how  people  comprehend,  monitor  and  remember  changes 
in  events  and  situations  in  the  world  based  on  the  information  they  receive.  Lastly,  we  are 
developing  a  computer  model  of  event  comprehension  and  memory  that  will  allow  for  a  much 
more  specific  prediction  of  a  given  individual’s  performance. 
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Potential  Army/Military  Applications 


The  potential  applications  of  this  research  for  the  Army  are  in  the  areas  of  training 
assessment  and  curriculum.  An  assessment  battery  measuring  complex  information 
comprehension  and  memory  may  be  used  to  identify  individuals  who  are  better  able  to  process 
complex  information  and  potentially,  to  identify  individuals’  areas  of  weakness.  These 
assessment  tools,  coupled  with  targeted  training,  can  serve  to  minimize  the  amount  of  time 
needed  to  increase  a  person’s  ability  to  comprehend  and  remember  event  information. 

Accomplishments 

We  have  completed  data  collection  exploring  the  accuracy  of  traditional  measures  of 
working  memory  abilities  and  their  prediction  of  performance  on  various  established  mental 
model  measures.  This  study  showed  that  while  these  measures  do  predict  a  few  aspects  of 
performance,  there  are  many  aspects  of  comprehension  and  memory  that  are  unrelated  to  them. 
Thus,  they  are  limited  in  their  ability  to  predict  future  performance.  In  many  cases,  other 
measures  of  complex  event  processing  were  better  at  predicting  future  performance.  The  data 
from  these  studies  has  been  broken  down  into  four  papers.  One  has  been  published  in  the  journal 
Memory  &  Cognition  where  it  was  the  lead  article,  one  is  under  review  at  the  Quarterly  Journal 
of  Experimental  Psychology ,  and  one  is  in  press  at  the  American  Journal  of  Psychology.  The 
fourth  manuscript  is  in  preparation  and  will  soon  be  submitted  to  the  Journal  of  Experimental 
Psychology:  Learning,  Memory  and  Cognition. 

Additionally,  we  are  continuing  to  develop  a  computer  simulation  of  mental  model 
construction  and  use.  To  constrain  its  initial  development,  we  have  limited  the  model  to  solving 
categorical  logic  problems.  To  date,  the  computer  model  has  been  successful  in  predicting  the 
accuracy  and  response  time  of  people’s  ability  to  understand  a  description  of  a  state  of  affairs  in 
the  world  and  in  drawing  the  appropriate  inferences  from  that  information.  We  will  continue  to 
expand  this  model  to  contain  other  sorts  of  events  knowledge.  This  model  is  a  very  exciting 
aspect  of  the  project  because  it  has  the  potential  to  identify,  with  a  high  degree  of  precision,  the 
specific  nature  of  the  difficulty  people  have  in  constructing  mental  models  and  making 
inferences  based  on  their  understanding  of  the  world. 

Final  Summary 

From  this  research,  there  are  a  number  of  key  findings.  First,  traditional  measures  of 
working  memory  span,  such  as  the  word  span  sentence  span,  operation  span,  and  spatial  span 
tests,  are  good  predictors  of  a  person’s  ability  to  remember  individual  pieces  of  information.  That 
is,  the  ability  to  remember  individual  facts.  However,  these  measures  are  unrelated  to  a  person’s 
ability  to  comprehend  and  remember  various  components  of  a  situation.  These  working  memory 
span  measures  only  influence  event  comprehension  and  memory  at  the  mental  model  level  to  the 
extent  that  thinking  at  that  level  depends  on  individual  facts.  If  a  person  can  go  beyond  this,  then 
their  performance  is  unrelated  to  working  memory  capacity.  The  situation-based  processes  that 
were  found  to  be  unrelated  to  working  memory  span  include  the  updating  of  event  knowledge, 
the  management  of  situation  information  retrieval,  the  detection  of  situational  inconsistencies, 
and  the  integration  of  multiple  pieces  of  information  that  refer  to  a  common  situation.  While 
there  was  little  to  no  relation  between  working  memory  span  and  event  cognition  performance, 
some  of  the  situation-based  cognitive  measures  were  related  to  one  another.  However,  because 


understanding  events  and  situations  involves  a  complex  of  many  different  types  of  information 
and  mental  processes,  the  inter-relation  of  these  components  was  limited. 
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Development  and  Empirical  Evaluation  of  a  Theory  of  Socially  Intelligent  Job 
Performance  for  United  States  Army  Junior  and  Non-Commissioned  Officers' 

Contract  #:  DASWO 1 -02-K-0002  Contract  Dates:  5/01/02- 1 0/1 5/03 

Institution:  Personnel  Decisions  Research  Institutes,  Inc.  PI:  Robert  J.  Schneider 

Research  Summary 

The  purpose  of  this  research  program  is  to  enhance  understanding  of  the  nature  of 
socially  competent  performance  in  junior  commissioned  military  officers.  This  involves  (1) 
determining  the  various  dimensions  of  socially  competent  performance;  (2)  identifying  the 
attributes  that  predict  those  social  performance  dimensions,  both  directly  and  indirectly  through 
other  attributes;  and  (3)  formulating  a  theory  of  how  various  attributes  (intelligence,  socially- 
oriented  personality  characteristics,  social  intelligence,  social  motivation,  and  social  knowledge) 
relate  to  social  performance,  and  to  each  other.  A  by-product  of  this  research  will  be  increased 
understanding  of  the  nature  of  social  knowledge  and  social  motivation.  Of  particular  interest  is 
whether  social  knowledge  and  social  motivation  each  consist  of  one  or  multiple  dimensions  and, 
if  they  consist  of  multiple  dimensions,  determining  the  nature  of  those  dimensions.  We  are 
carrying  out  this  research  because  of  the  increasing  importance  of  social  competence  to  the 
Army’s  ability  to  carry  out  its  missions.  For  example,  an  officer's  social  competence  helps 
determine  his  or  her  ability  to  foster  unit  cohesion,  mentor  soldiers,  work  effectively  with 
individuals  ranging  widely  in  personality  and  work  style,  lead  effectively  when  deployed  to 
foreign  countries,  and  handle  new  roles  (e.g.,  promotions)  effectively. 

Contributions  to  Basic  Research 

This  research  advances  ongoing  theoretical  work  by  a  number  of  researchers  on  the 
nature  and  predictors  of  effective  job  performance.  It  does  so  by  (1)  focusing  in  on  the  major 
dimensions  of  social  job  performance;  (2)  determining  the  dimensions  of  social  knowledge  and 
social  motivation,  attributes  believed  to  directly  influence  effective  social  job  performance;  and 
(3)  formulating  and  testing  separate  theories  (all  with  the  same  general  form)  of  the  attributes 
that  predict  each  major  dimension  of  social  job  performance.  Our  research  will  also  provide 
additional  insight  into  the  relationship  between  intelligence  and  social  abilities,  a  research  topic 
that  has  been  of  keen  interest  to  researchers  since  the  1920s,  when  social  intelligence  was  first 
researched,  and  which  remains  of  interest  today. 

Another  important  scientific  contribution  of  this  research  involves  the  manner  in  which 
social  knowledge  is  measured.  The  social  knowledge  test  (SKT)  used  in  this  research  involves  a 
series  of  video-based  social  scenarios  in  which  actors  portray  socially  effective  and  socially 
ineffective  behaviors.  People's  social  knowledge  scores  are  based  on  the  number  of  targeted 
effective  and  ineffective  social  behaviors  they  are  able  to  identify  correctly  (including  reasons 


1  We  had  initially  hoped  to  utilize  actual  Army  junior  commissioned  officers  and  NCOs  as  subject  matter  experts 
and  examinees  in  this  research.  However,  it  was  not  possible  to  gain  access  to  these  types  of  individuals  in  a  way 
that  was  consistent  with  our  research  needs.  We  therefore  made  use  of  advanced  (3rd  and  4th  year)  ROTC  cadets  and 
midshipmen.  Since  ROTC  cadets  and  midshipmen  are  in  training  to  become  junior  commissioned  officers,  and  not 
NCOs,  they  were  not  in  a  position  to  speak  to  the  requirements  of  NCO  positions.  The  population  to  which  our 
project  will  be  generalized  was  therefore  necessarily  changed  to  junior  commissioned  officers. 
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why  they  are  effective  or  ineffective).  In  this  way,  a  long-standing  criticism  that  social 
intelligence/knowledge  tests  fail  to  use  testing  formats  that  are  sufficiently  "social"  in  nature  is 
addressed.  This  SKT  also  requires  people  to  identify  socially  effective  and  ineffective  behaviors 
from  an  ongoing  stream  of  social  behavior  rather  than  to  pick  the  best  response  to  a  social 
situation  from  among  a  list  of  possible  responses  that  are  provided  to  them.  Arguably,  then,  our 
approach  to  testing  social  knowledge  is  more  representative  of  "real  life"  than  other  testing 
approaches. 


Potential  Army/Military  Applications 

There  are  several  possible  ways  this  basic  research  can  be  applied  to  address  important 
Army/military  needs.  One  would  involve  use  of  the  social  performance  measure  we  have 
developed  by  supervisors  or  mentors  to  develop  junior  commissioned  officers  on  the  dimensions 
of  social  job  performance  that  emerged  in  our  research.  The  social  performance  measure  could 
also  be  adapted  for  use  as  a  self-development  tool  that  could  be  administered  on  the  Army’s 
Intranet.  Specific  developmental  suggestions  for  each  social  performance  dimension  could  be 
created.  Developmental  suggestions  could  then  be  presented  for  those  social  performance 
dimensions  on  which  an  officer  needs  to  improve,  based  on  his/her  self-assessed  social  job 
performance.  This  would  have  the  virtue  of  providing  a  mechanism  for  development  of 
interpersonal/leadership  skills  in  a  dispersed,  web-based  environment.  Another  possible 
application  would  involve  adaptation  of  the  SKT  for  training  purposes.  Specifically,  the  SKT 
could  be  used  as  a  training  needs  assessment  tool  to  determine  an  officer’s  social  knowledge 
deficits.  The  scoring  guidelines  we  have  created  for  the  SKT  could  then  be  adapted  as  learning 
points  to  be  discussed  either  with  a  supervisor/mentor  or  in  a  group  format  with  a 
trainer/facilitator. 

This  research  also  has  applications  to  selection/promotion.  In  evaluating  our  theory,  we 
will  be  determining  the  extent  to  which  measures  of  various  attributes  correlate  with  various 
dimensions  of  social  job  performance.  Some  of  these  attributes  (e.g.,  social  intelligence,  social 
planning  ability,  charisma,  social  motivation)  may  not  currently  be  used  in  the  Army’s  current 
selection/promotion  procedures  for  its  junior  commissioned  officers.  This  research  could 
therefore  be  used  to  develop  composites  that  may  increase  the  accuracy  with  which  the 
effectiveness  of  military  officers  can  be  predicted.  And  while  the  SKT  is  somewhat  lengthy  to 
administer,  in  principle,  it  could  be  used  as  a  selection/promotion  tool  as  well  as  a  training  needs 
assessment  tool,  assuming  our  research  shows  it  to  be  valid. 

Finally,  the  20  social/leadership  scenarios  in  the  SKT  could  provide  a  foundation  for 
several  other  assessment  tools  to  be  developed  in  the  future.  These  tools  might,  for  example, 
include  a  social  skills  measure,  in  which  the  action  in  the  scenario  is  stopped  at  a  critical  point 
and  the  person  being  tested  is  asked  to  assume  the  role  of  one  of  the  characters  in  the  scenario. 
That  individual’s  social  performance  in  his/her  assigned  role  could  then  be  videotaped  (or  simply 
observed)  and  scored  using  scoring  guidelines  to  be  developed  in  the  future. 
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Accomplishments 


Our  accomplishments  to  date  are  as  follows: 

■  Developed  taxonomy  of  social  job  performance  requirements  relevant  to  junior 
commissioned  officers.  This  involved  extensive  review  of  relevant  literature  and  past 
PDRI  project  work,  as  well  as  a  research  study  in  which  PDRI  psychologists  sorted 
examples  of  social  job  performance  that  emerged  from  that  review  into  similar 
categories.  Results  of  this  sorting  task  were  then  analyzed  using  statistical  methods  to 
develop  a  two-level  taxonomy  of  social  job  performance  requirements.  The  final 
taxonomy  included  7  broad  dimensions  and  20  more  specific  facets  falling  within 
those  various  dimensions. 

■  Developed  and  pilot  tested  social  performance  measure  based  on  that  taxonomy.  We 
developed  a  52-item  social  performance  measure  that  can  be  completed  by  officers’ 
superiors,  peers,  and/or  subordinates  (or,  potentially,  by  the  officers  themselves).  It 
measures  the  6  social  performance  dimensions  and  1 7  social  performance  facets  from 
our  social  performance  taxonomy  on  which  ROTC  cadets  can  be  rated  (since  they 
comprise  the  group  on  which  our  theory  will  be  tested).  This  social  performance 
measure  also  includes  several  rating  tips  so  that  common  errors  made  by  raters  (e.g., 
halo,  leniency)  can  be  minimized.  The  social  performance  measure  was  pilot  tested 
on  a  small  sample  of  ROTC  cadre  officers  to  help  ensure  its  accuracy  and  relevance 
to  3  rd  and  4th  year  ROTC  cadets. 

■  Developed  and  pilot  tested  SKT.  Building  on  our  social  job  performance  taxonomy 
work,  we  developed  and  pilot  tested  a  26-scenario,  video-based  SKT.  Based  on 
results  of  a  pilot  test  involving  22  ROTC  cadets,  we  chose  20  scenarios  for  the 
version  of  the  SKT  that  will  be  used  to  test  our  theory.  Initial  pilot  test  results  were 
promising.  Scores  on  the  SKT  were  somewhat  low.  However,  a  substantial 
percentage  of  the  targeted  scoring  criteria  appear  to  be  working  well,  and  the  range  of 
scores  obtained  by  the  pilot  test  participants  revealed  that  they  varied  substantially  in 
their  SKT  scores.  The  SKT  that  we  developed  is  presented  on  videocassettes,  and 
includes  detailed,  carefully  developed  scoring  guidelines. 

■  Developed  self-report  social  motivation  measure.  We  wrote  49  items  that  measure 
motivational  aspects  of  the  social  knowledge/performance  dimensions  assessed  in  the 
20  social  scenarios  comprising  the  SKT. 

Future  Plans 

Our  intent  is  to  administer  a  battery  of  measures  assembled  to  test  our  theory  of  socially 
competent  job  performance  to  a  sample  of  approximately  150  3rd  and  4th  year  ROTC  cadets 
and/or  midshipmen.  This  will  be  done  either  during  Fall,  2003,  at  several  universities  with 
ROTC  programs,  or  during  the  Summer  of  2004  using  ROTC  cadets  attending  the  National 
Advanced  Leadership  Camp  at  Fort  Lewis,  Washington  (with  the  latter  requiring  an  extension  of 
the  Project’s  time  frame).  We  anticipate  completion  of  the  project  within  six  weeks  after  the 
data  have  been  collected.  Social  performance  measures  will  most  likely  be  collected  from  peers 
if  data  are  collected  at  universities  because  Navy  and  Air  Force  ROTC  cadre  officers  would, 
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most  likely,  not  be  in  a  position  to  rate  important  aspects  of  the  social  performance  of  their 
cadets  accurately  due  to  limited  opportunity  to  observe  such  behavior.  If,  on  the  other  hand,  data 
are  collected  at  the  National  Advanced  Leadership  Camp,  we  will  most  likely  collect  social 
performance  measures  from  ROTC  cadre  officers  serving  as  trainers  since  they  will  have  had  an 
opportunity  to  closely  observe  the  cadets’  social  behavior  over  a  substantial  period  of  time  (about 
a  month). 
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Social  Structures  Affecting  Army  Performance 


Contract  #:  DASW  0100  K  0016  Contract  Dates:  8/17/2000-  8/16/2004 

Institution:  University  of  Maryland  at  Pis:  David  R.  Segal  and  Mady  Wechsler 

College  Park  Segal 


Research  Summary 

The  purpose  of  this  research  is  the  identification  and  analysis  of  changes  in  the 
international  environment,  in  American  society,  in  the  U.S.  Army,  and  in  the  American  soldier, 
that  are  consequential  for  the  fulfillment  of  Army  missions.  The  research  program  consists  of 
multiple  projects  organized  into  four  inter-related  areas:  (a)  research  on  American  populations, 
(b)  research  on  soldiers  and  their  families,  (c)  diversity  in  the  military,  and  (d)  comparative 
military  institutions.  Most  of  our  research  on  American  populations  has  focused  on  the  youth 
population  from  which  the  Army’s  recruits  are  drawn.  In  particular,  collaborating  with 
colleagues  at  the  University  of  Michigan,  we  utilize  the  Monitoring  the  Future  (MtF)  database. 
This  research  program  surveys  about  16,000  high  school  seniors  each  spring  semester  and  has 
been  doing  so  since  1975,  producing  a  rich  archive  of  data  on  American  youth.  Sub-samples  of 
seniors  are  then  surveyed  semi-annually  for  several  years  after  high  school  graduation.  Among 
the  areas  covered  in  these  surveys  are  military-related  attitudes  and  behaviors.  In  our  research 
program  on  soldiers  and  their  families  we  conduct  surveys  of  soldiers  in  selected  units  and 
surveys  of  military  wives.  We  also  conduct  interviews  with  soldiers  and  spouses,  conduct 
observations  of  units  in  the  field,  and  draw  upon  documentary  materials  in  a  variety  of  archives. 
In  the  area  of  diversity,  our  concern  is  with  variations  in  adaptation  to  the  military  and  its 
missions  as  they  are  affected  by  race,  ethnicity,  gender,  and  sexual  orientation.  We  draw  upon 
our  own  surveys  and  observations,  secondary  analysis  of  other  surveys,  and  field  observations. 
Regarding  comparative  military  institutions,  our  goal  is  to  draw  upon  the  experiences  of  foreign 
military  forces  to  identify  cross-national  similarities  and  differences  as  well  as  to  learn  from  the 
experiences  of  others.  Our  approaches  have  been  to  encourage  cross-national  groups  of 
behavioral  scientists  to  engage  in  parallel  research  efforts  in  their  own  countries,  to  pool  their 
knowledge,  and  to  conduct  surveys  and  interviews  within  the  armed  forces  of  other  nations. 


Contributions  to  Basic  Research 

Our  research  on  American  youth  has  contributed  to  our  understanding  of  attitudes  and 
behaviors  at  the  important  life-course  transition  point  between  adolescence  and  adulthood,  and 
particularly  to  our  understanding  of  why  some  high  school  graduates  elect  to  enter  the  military 
rather  than  following  other  trajectories,  such  as  college  or  civilian  employment.  Our  research  on 
soldiers  and  their  families  has  allowed  us  to  use  the  military  as  a  field  laboratory  to  study  and 
understand  the  range  of  tensions  that  exist  at  the  interface  between  work  life  and  family  life,  and 
the  role  that  the  family  plays  in  work  performance,  satisfaction,  and  retention.  Our  research  on 
diversity  increases  our  understanding  of  how  an  organization  that  has  traditionally  drawn  on  the 
dominant  majority  sector  of  American  society  can  adapt  to  the  changing  demographics  of  the 
American  labor  force,  as  we  move  toward  a  population  in  which  no  group  constitutes  a  majority. 
Our  comparative  research  program,  focusing  on  similarities  and  differences  in  military  forces 
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cross-nationally,  contributes  at  the  micro-level  to  our  understanding  of  how  soldiers  (and  other 
employees)  adapt  to  such  life-course  experience  as  organizational  down-sizing,  at  the  meso-level 
to  our  understanding  of  how  processes  of  organizational  change  take  place,  and  at  the  macro¬ 
level  to  our  understanding  of  how  general  social  processes  such  as  globalization  affect  military 
organization. 


Potential  Army/Military  Applications 

Our  research  program  on  the  attitudes  and  behavior  of  American  youth  is  relevant  for 
understanding  the  labor  market  environment  within  which  the  Army  recruiting  process  takes 
place.  The  U.S.  Army  Recruiting  Command  has  used  the  results  of  this  research  for  several 
years,  and  it  was  recently  cited  in  a  National  Research  Council  study  on  the  youth  population 
conducted  for  the  Office  of  the  Secretary  of  Defense. 

Our  research  program  on  Army  families  has  contributed  to  the  development  of  ARI 
handbooks  for  family  support  groups  and  has  assisted  Army  leaders  who  must  provide  support  to 
families  during  deployments.  This  family  research  program  was  recently  discussed  with  LTG 
LeMoyne  (Army  G-l)  and  LTG  Cody  (Army  G-3)  with  regard  to  its  implications  for  the  Army 
Transition.  More  recently  it  was  discussed  with  Col.  Mike  Pfenning  for  its  relevance  to  the 
Army  Well-Being  Program.  Much  of  our  research  on  soldiers  addresses  adaptation  to  peace 
operations  and  is  applicable  to  preparing  units  for  such  missions.  Dr.  David  Segal  has  been 
called  on  frequently  to  make  Officer  Professional  Development  presentations  to  the  leaders  of 
units  scheduled  for  deployment  on  peacekeeping  missions.  He  was  also  recently  invited  to  meet 
with  the  Reserve  Forces  Policy  Board  to  discuss  the  role  of  reserves  in  contemporary  missions. 

Our  research  on  diversity,  and  particularly  on  gender  integration,  has  been  used  in 
ongoing  policy  discussions  on  the  role  of  women  in  the  military.  For  example,  Dr.  Mady  Segal, 
who  was  a  member  of  the  Commission,  presented  our  work  to  the  Congressional  Commission  on 
Military  Training  and  Gender-Related  Issues.  This  influenced  the  decision  by  a  majority  of  the 
Commissioners  to  recommend  to  the  Congress  that  the  Army  be  allowed  to  maintain  its  current 
pattern  of  gender  integration  in  entry-level  training.  Mady  and  David  Segal  also  have  been 
consulted  regarding  sexual  abuse  problems  at  the  U.S.  Air  Force  Academy. 

Our  comparative  research  program  serves  as  a  source  of  lessons  learned  by  variations  in 
military  organization.  For  example,  David  R.  Segal  has  been  asked  to  address  the  participants  in 
an  OSD  summer  study  on  the  future  of  the  officer  corps  in  July  2003.  He  will  speak  about 
variations  in  ways  in  which  military  forces  access,  train,  and  educate  their  officers,  and  in 
relations  between  officers  and  enlisted  personnel. 

Accomplishments 

Our  research  on  the  American  youth  population  has  demonstrated  a  strong  relationship 
between  propensity  to  serve  in  the  military  and  actual  enlistment,  but  also  a  long-term  declining 
trend  in  propensity  to  serve.  We  have  demonstrated  racial,  ethnic,  and  gender  differences  in 
propensity,  and  in  the  relationship  between  propensity  and  enlistment.  Most  recently,  we  have 
shown  a  decline  in  perceptions  that  the  military  is  a  favorable  work  environment  for  women  and 
minorities.  Our  data  have  also  been  used  to  address  the  hypothesis,  widely  discussed  in  the 
1 990s,  that  there  was  a  “culture  gap”  between  the  American  military  and  American  society.  Our 
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analyses  show  that  the  people  who  enter  the  military  are  in  the  mainstream  of  American  society 
in  terms  of  their  attitudes. 

Our  research  on  soldiers  focused  primarily  on  personnel  from  the  1 0th  Mountain  Division 
and  from  the  Reserve  Components  who  were  deployed  on  peacekeeping  missions.  We  found 
that  deployment  had  a  negative  impact  on  morale,  but  that  multiple  deployments  did  not  reduce 
reenlistment  intentions.  We  also  found  that  participation  in  peacekeeping  did  not  decrease  the 
martial  mindset  of  American  soldiers. 

In  our  research  on  women  in  the  military,  we  have  found  that  social  values  outweigh 
empirical  evidence  in  public  discourse,  with  certain  arguments  about  gender  integration 
persisting  within  new  contexts  despite  evidence  from  research  and  military  experience  refuting 
the  arguments.  With  regard  to  military  families,  we  found  that  the  effects  of  moving  on  military 
spouse  employment  vary  by  the  rank  of  the  soldier  and  the  race  and  gender  of  the  spouse.  We 
have  also  found  that  first-term  soldiers  (in  a  small  sample  of  infantry  soldiers)  tend  to  view 
themselves  in  terms  of  their  family  identities  more  than  in  terms  of  their  identities  as  soldiers. 

Future  Plans 

We  will  be  working  on  a  number  of  projects  during  the  next  year.  Our  research  on 
American  populations  will  focus  on  a  series  of  surveys  at  the  U.S.  Naval  Academy,  using 
modified  versions  of  the  Monitoring  the  Future  research  instruments.  We  will  study  the  ways  in 
which  high  school  graduates  who  attend  a  military  academy  are  similar  to  or  different  from  their 
peers  who  attend  civilian  institutions  of  higher  education  in  terms  of  their  attitudes  and  values. 
We  will  also  study  the  changes  that  take  place  during  initial  plebe  socialization  and  during  the 
four  years  of  academy  education.  We  will  also  be  using  the  Monitoring  the  Future  data  bank  to 
explore  changes  in  the  perceptions  of  the  gender  and  racial  climates  of  the  American  military  by 
high  school  seniors. 

Our  research  on  American  soldiers  will  involve  the  analysis  of  data  we  collected  last  year 
from  first-term  soldiers  in  two  of  the  Army’s  interim  brigades,  focusing  on  the  relationship 
between  their  former  propensity  to  serve  and  motivations  for  service,  and  on  the  process  of 
developing  an  identity  as  a  soldier.  We  are  also  preparing  a  monograph  on  the  demography  of 
the  American  military  to  be  published  by  Population  Bulletin. 

During  the  next  year,  our  research  in  military  families  will  include  a  study  of  military  and 
civilian  families’  economic  behavior  and  financial  well  being  and  an  analysis  of  military  family 
research  (as  a  chapter  in  a  book  whose  theme  is  how  research  designed  to  meet  national  security 
needs  was  an  integral  part  of  the  evolution  of  psychology).  We  will  be  looking  particularly  at  the 
implications  of  tied  migration — the  fact  that  the  military  relocates  not  only  soldiers  but  also 
families — on  the  economic  well  being  of  military  families. 

Our  major  effort  in  the  diversity  area  will  be  a  book  that  Mady  Segal  is  writing  to  capture 
findings  from  her  thirty  years  of  research  on  the  integration  of  women  in  the  military,  involving 
synthesis  of  theory  and  research,  analysis  of  data  on  basic  training  collected  by  the 
Congressional  Commission  on  Military  Training  and  Gender-Related  Issues,  analysis  of  public 
discourse  on  women’s  involvement  in  military  activities,  and  analysis  of  the  public  policy 
formation  process. 
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With  regard  to  comparative  military  institutions,  two  efforts  are  underway.  We  are  in  the 
process  of  collecting  data  on  members  of  the  Japan  Self-Defense  Force  who  have  participated  in 
UN  peacekeeping  missions  to  increase  our  understanding  of  how  personnel  who  historically  have 
not  been  peacekeepers  (in  this  case  because  they  were  precluded  from  out-of-area  missions) 
adapt  to  this  new  role.  We  are  also  preparing  an  analysis  of  our  Russian  army  downsizing  data 
for  presentation  at  a  conference  on  the  life  course  at  Pennsylvania  State  University  in  the  Fall. 
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Construct  Validating  Aspects  of  the  Theory  of  Successful  Intelligence  via  a 
Test  Battery  for  Measuring  Mental  Flexibility 

Contract  #:  DASW01-03-K-0001  Contract  Dates:  Mar-0 1-03  to  Feb-28-06 

Institution:  Yale  University  PI:  Robert  J.  Sternberg 

Research  Summary 

Robert  J.  Sternberg’s  triarchic  theory  of  successful  intelligence  provides  a  broader 
conceptualization  of  intelligence  than  what  is  typically  assumed  by  standard  intelligence  tests. 
The  triarchic  theory  comprises  three  subtheories  that  deal  with  distinct  but  related  aspects  of 
human  functioning.  First,  the  componential  subtheory  deals  with  the  more  universal  components 
of  intelligence.  These  are  the  types  of  mental  processes  that  intelligence  tests  have  traditionally 
assessed.  Second,  the  contextual  subtheory  deals  with  the  types  of  mental  processes  that  are 
needed  to  successfully  adapt  to  the  environment,  to  change  (shape)  the  environment,  and  to 
select  new  environments  if  adaptation  and  shaping  are  not  appropriate.  And  third,  the 
experiential  subtheory  deals  with  mental  flexibility  —  the  ability  to  cope  with  novelty  and  to 
make  routine  processing  more  automatic.  Mental  flexibility  plays  a  critical  role  in  the  successful 
application  of  the  mental  processes  needed  for  adapting  to,  selecting,  and  shaping  the 
environment  in  novel  situations. 

The  goal  of  this  research  is  to  develop  a  multifaceted  test  of  flexible  thinking  that 
measures  how  well  one  can  apply  the  components  of  intelligence  in  relatively  novel  tasks  and 
situations.  Two  basic  paradigms  for  assessing  mental  flexibility  are  being  investigated.  The  first 
paradigm  conceptualizes  flexibility  from  the  perspective  of  the  experiential  subtheory,  as  just 
described.  The  test  battery  that  will  be  created  from  this  theoretical  foundation  will  be  validated 
against  known  measures  of  abilities  that  have  predictable  relationships  with  mental  flexibility. 
The  second  is  a  “sensitivity”  paradigm,  which  proposes  an  alternative  conceptualization  of 
mental  flexibility.  This  alternative  paradigm  views  mental  flexibility  as  a  person’s  sensitivity  to 
subtle  regularities  that  can  be  manipulated  in  the  environment  (or  task).  The  research  will  be 
implemented  in  two  phases.  The  first,  formative  phase  will  further  develop  and  refine  the  tasks 
and  methodologies  in  each  paradigm.  The  second,  summative  phase,  will  investigate  the  nature 
of  mental  flexibility  using  the  final  test  battery  and  refined  methodologies. 

Contributions  to  Basic  Research 

The  current  research  advances  basic  knowledge  in  the  areas  of  intelligence  and  creativity, 
and  helps  to  clarify  the  theoretical  understanding  of  the  mental  processes  involved  in  flexible 
thinking.  The  theory  of  successful  intelligence  differs  somewhat  from  conventional  theories  of 
intelligence  in  its  conceptualization  of  what  mental  flexibility  is  and  where  mental  flexibility 
belongs  in  a  theory  of  intelligence.  The  experiential  subtheory  of  successful  intelligence 
conceptualizes  mental  flexibility  as  a  part  of  creative  thinking,  but  not  the  only  part.  That  is, 
while  creative  intelligence  requires  mental  flexibility  as  an  antecedent  for  convergent  and 
divergent  thinking,  mental  flexibility  alone  is  not  sufficient  for  creativity.  The  contribution  of 
this  research  to  basic  science  will  be  to  clarify  (1)  the  nature  of  the  cognitive  processes 
underlying  mental  flexibility,  (2)  the  place  of  mental  flexibility  in  the  experiential  subtheory,  and 
(3)  mental  flexibility’s  relationship  with  other  aspects  of  intellectual  functioning  (such  as 
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convergent  and  divergent  thinking,  analytical  and  practical  thinking,  as  well  as  traditionally 
assessed  aspects  of  intelligence  like  abstract  and  verbal  reasoning  skills). 

Potential  Army/Military  Applications 

A  test  of  mental  flexibility  that  assesses  competencies  that  fall  outside  of  the  framework 
used  by  conventional  tests  of  intelligence  is  potentially  valuable  for  both  selection  and  training. 

Selection:  Making  promotion  decisions  at  high  levels  of  rank  often  means  selecting  from 
a  group  of  people  that  are  relatively  homogeneous  with  regard  to  general  intellectual  abilities.  If 
mental  flexibility  is  truly  distinguishable  from  general  intelligence,  then  it  may  prove  to  be  a 
critical  factor  in  optimizing  promotion  decisions.  Clarifying  the  link  between  mental  flexibility 
and  creativity  has  added  potential  to  identify  people’s  capacity  for  innovation  -  an  important 
skill  highly  desired  in  senior  officers. 

Training:  The  importance  of  flexible  thinking  is  evident  in  the  function  of  military 
personnel  at  all  levels.  In  the  field,  leaders  need  to  adapt  quickly  to  the  new  tactics  of  their 
opponents,  to  deal  with  changing  commands  from  superiors,  and  to  cope  with  unstable  and 
unanticipated  environments.  Troops  need  to  implement  new  orders  quickly  and  effectively 
without  being  rigidly  tied  to  previous  ways  of  doing  things.  On  the  base,  both  leaders  and  troops 
need  to  accommodate  constant  changes  in  personnel  and  varying  levels  of  ongoing  change  in  the 
organizational  structure.  Research  that  builds  on  our  understanding  of  the  cognitive  processes 
entailed  in  flexible  thinking  can  be  used  to  develop  effective  instructional  programs  that  facilitate 
the  use  of  flexible  thinking  strategies  in  a  variety  of  contexts. 

Accomplishments 

Given  this  is  a  new  project,  our  major  activities  to  date  have  been  in  preparation  for  data 
collection  to  begin  in  the  fall  of  2003.  We  have  recruited  at  the  research  assistant  level  to  assist 
with  this  and  are  also  in  the  process  of  recruiting  a  software  developer.  We  are  currently 
preparing  the  initial  pool  of  test  items  to  be  programmed  ready  for  piloting  and  are  reviewing  the 
theoretical  avenues  suggested  by  the  ARI  research  team  at  the  May  13,  2003  ARI  kickoff 
meeting  in  Washington  DC. 


Future  Plans 

Our  plans  for  the  next  year  are  to  continue  with  the  first  phase  of  the  project.  We  intend 
to  (1)  continue  with  the  preparation  of  the  test  battery,  (2)  begin  implementation  and  data 
collection  for  the  formative  phase  of  the  research;  and  (3)  further  develop  the  theoretical 
specifications  of  the  alternative  “sensitivity”  paradigm.  Based  on  the  result  of  this  research  and 
our  pilot  studies,  we  will  modify,  as  needed,  existing  tasks  and  approaches  in  preparation  for  the 
summative  second  phase  of  the  research. 
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